{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作业题目：xgboost参数调优"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "解题提示：\n",
    "为减轻大家对特征工程的入手难度，以及统一标准，数据请用课程网站提供的特征工程编码后的数据（RentListingInquries_FE_train.csv）或稀疏编码的形式（RentListingInquries_FE_train.bin）。xgboost既可以单独调用，也可以在sklearn框架下调用。大家可以随意选择。若采用xgboost单独调用使用方式，建议读取稀疏格式文件。\n",
    "\n",
    "作业要求：\n",
    "独立调用xgboost或在sklearn框架下调用均可。\n",
    "1. 模型训练：超参数调优\n",
    "a) 初步确定弱学习器数目： 20分\n",
    "b) 对树的最大深度（可选）和min_children_weight进行调优（可选）：20分\n",
    "c) 对正则参数进行调优：20分\n",
    "d) 重新调整弱学习器数目：10分\n",
    "e) 行列重采样参数调整：10分\n",
    "2. 调用模型进行测试10分\n",
    "3. 生成测试结果文件10分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.导入必要的包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import log_loss\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>3000</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>750.000000</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>5465</td>\n",
       "      <td>2732.5</td>\n",
       "      <td>1821.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.0</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>3275</td>\n",
       "      <td>1637.5</td>\n",
       "      <td>1637.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 225 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  latitude  longitude  price  price_bathrooms  \\\n",
       "0        1.5         3   40.7145   -73.9425   3000           1200.0   \n",
       "1        1.0         2   40.7947   -73.9667   5465           2732.5   \n",
       "2        1.0         1   40.7388   -74.0018   2850           1425.0   \n",
       "3        1.0         1   40.7539   -73.9677   3275           1637.5   \n",
       "4        1.0         4   40.8241   -73.9493   3350           1675.0   \n",
       "\n",
       "   price_bedrooms  room_diff  room_num  Year       ...        walk  walls  \\\n",
       "0      750.000000       -1.5       4.5  2016       ...         0.0    0.0   \n",
       "1     1821.666667       -1.0       3.0  2016       ...         0.0    0.0   \n",
       "2     1425.000000        0.0       2.0  2016       ...         0.0    0.0   \n",
       "3     1637.500000        0.0       2.0  2016       ...         0.0    0.0   \n",
       "4      670.000000       -3.0       5.0  2016       ...         0.0    0.0   \n",
       "\n",
       "   war  washer  water  wheelchair  wifi  windows  work  interest_level  \n",
       "0  0.0     0.0    0.0         0.0   0.0      0.0   0.0             1.0  \n",
       "1  0.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "2  0.0     0.0    0.0         0.0   0.0      0.0   0.0             0.0  \n",
       "3  0.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "4  1.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "\n",
       "[5 rows x 225 columns]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# path to where the data lies\n",
    "#dpath = './data/'\n",
    "train = pd.read_csv(\"RentListingInquries_FE_train.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>96561</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>40.7426</td>\n",
       "      <td>-73.9790</td>\n",
       "      <td>3200</td>\n",
       "      <td>1600.0</td>\n",
       "      <td>1066.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96562</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40.7102</td>\n",
       "      <td>-74.0163</td>\n",
       "      <td>3950</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>1975.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96563</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40.7601</td>\n",
       "      <td>-73.9900</td>\n",
       "      <td>2595</td>\n",
       "      <td>1297.5</td>\n",
       "      <td>1297.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96564</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>40.7066</td>\n",
       "      <td>-74.0101</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>3350.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96565</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>40.8699</td>\n",
       "      <td>-73.9172</td>\n",
       "      <td>2200</td>\n",
       "      <td>1100.0</td>\n",
       "      <td>733.333333</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 225 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       bathrooms  bedrooms  latitude  longitude  price  price_bathrooms  \\\n",
       "96561        1.0         2   40.7426   -73.9790   3200           1600.0   \n",
       "96562        1.0         1   40.7102   -74.0163   3950           1975.0   \n",
       "96563        1.0         1   40.7601   -73.9900   2595           1297.5   \n",
       "96564        1.0         0   40.7066   -74.0101   3350           1675.0   \n",
       "96565        1.0         2   40.8699   -73.9172   2200           1100.0   \n",
       "\n",
       "       price_bedrooms  room_diff  room_num  Year       ...        walk  walls  \\\n",
       "96561     1066.666667       -1.0       3.0  2016       ...         0.0    0.0   \n",
       "96562     1975.000000        0.0       2.0  2016       ...         0.0    0.0   \n",
       "96563     1297.500000        0.0       2.0  2016       ...         0.0    0.0   \n",
       "96564     3350.000000        1.0       1.0  2016       ...         0.0    0.0   \n",
       "96565      733.333333       -1.0       3.0  2016       ...         0.0    0.0   \n",
       "\n",
       "       war  washer  water  wheelchair  wifi  windows  work  interest_level  \n",
       "96561  0.0     0.0    0.0         0.0   0.0      0.0   0.0             1.0  \n",
       "96562  0.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "96563  1.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "96564  1.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "96565  0.0     0.0    0.0         0.0   0.0      0.0   0.0             2.0  \n",
       "\n",
       "[5 rows x 225 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 96566 entries, 0 to 96565\n",
      "Columns: 225 entries, bathrooms to interest_level\n",
      "dtypes: float64(23), int64(202)\n",
      "memory usage: 165.8 MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "习惯性看一下数据的开头最后和相关信息，并没有发现什么异常"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bathrooms          0\n",
       "bedrooms           0\n",
       "latitude           0\n",
       "longitude          0\n",
       "price              0\n",
       "price_bathrooms    0\n",
       "price_bedrooms     0\n",
       "room_diff          0\n",
       "room_num           0\n",
       "Year               0\n",
       "Month              0\n",
       "Day                0\n",
       "Wday               0\n",
       "Yday               0\n",
       "hour               0\n",
       "top_10_manager     0\n",
       "top_25_manager     0\n",
       "top_5_manager      0\n",
       "top_50_manager     0\n",
       "top_1_manager      0\n",
       "top_2_manager      0\n",
       "top_15_manager     0\n",
       "top_20_manager     0\n",
       "top_30_manager     0\n",
       "1br                0\n",
       "24                 0\n",
       "2br                0\n",
       "3br                0\n",
       "ac                 0\n",
       "access             0\n",
       "                  ..\n",
       "sublet             0\n",
       "subway             0\n",
       "super              0\n",
       "superintendent     0\n",
       "swimming           0\n",
       "tenant             0\n",
       "term               0\n",
       "terrace            0\n",
       "time               0\n",
       "tons               0\n",
       "train              0\n",
       "true               0\n",
       "tv                 0\n",
       "unit               0\n",
       "utilities          1\n",
       "valet              1\n",
       "video              1\n",
       "view               1\n",
       "views              1\n",
       "virtual            1\n",
       "walk               1\n",
       "walls              1\n",
       "war                1\n",
       "washer             1\n",
       "water              1\n",
       "wheelchair         1\n",
       "wifi               1\n",
       "windows            1\n",
       "work               1\n",
       "interest_level     1\n",
       "Length: 225, dtype: int64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看一下数据集中是否又空值\n",
    "train.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.dropna(inplace=True)\n",
    "#查看一下数据集中是否又空值\n",
    "#train.isnull().sum()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "习惯性看一下有没有空值，发现里面部分特征有一个缺失值，因为数据量较大，所以我们直接将其所在行删去。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = train['interest_level']\n",
    "\n",
    "train = train.drop([\"interest_level\"],axis = 1)\n",
    "X_train = train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# 训练样本6w+，交叉验证太慢，用train_test_split估计模型性能\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part, X_val, y_train_part, y_val = train_test_split(X_train, y_train, train_size = 0.33,random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第一步：调用内嵌cv寻找n_estimators的最佳参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#直接调用xgboost内嵌的交叉验证（cv），可对连续的n_estimators参数进行快速交叉验证\n",
    "#而GridSearchCV只能对有限个参数进行交叉验证\n",
    "def modelfit(alg, X_train, y_train, cv_folds=None, early_stopping_rounds=10):\n",
    "    xgb_param = alg.get_xgb_params()\n",
    "    xgb_param['num_class'] = 9\n",
    "    \n",
    "    #直接调用xgboost，而非sklarn的wrapper类\n",
    "    #准备数据\n",
    "    xgtrain = xgb.DMatrix(X_train, label = y_train)\n",
    "        \n",
    "    #调用cv\n",
    "    cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], folds =cv_folds,\n",
    "             metrics='mlogloss', early_stopping_rounds=early_stopping_rounds)\n",
    "  \n",
    "    cvresult.to_csv('1_nestimators.csv', index_label = 'n_estimators')\n",
    "    \n",
    "    #最佳参数n_estimators\n",
    "    n_estimators = cvresult.shape[0]\n",
    "    \n",
    "    # 采用交叉验证得到的最佳参数n_estimators，训练模型\n",
    "    alg.set_params(n_estimators = n_estimators)\n",
    "    alg.fit(X_train, y_train, eval_metric='mlogloss')\n",
    "        \n",
    "    #Predict training set:\n",
    "    #train_predprob = alg.predict_proba(X_train)\n",
    "    #logloss = log_loss(y_train, train_predprob)\n",
    "\n",
    "   #Print model report:\n",
    "   # print (\"logloss of train :\" )\n",
    "   # print logloss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "#params = {\"objective\": \"multi:softprob\", \"eval_metric\":\"mlogloss\", \"num_class\": 9}\n",
    "\n",
    "xgb1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #数值大没关系，cv会自动返回合适的n_estimators\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel=0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb1, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEXCAYAAABCjVgAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucJHV97//Xuy9z2zvsLLvscpWLIEHUVUCjEi8JEgNGCYqiQogc89OjJnqMejyamJhoLkaNRoMK/IwK8UIUicoxKBKNIIsgV9GFXdiFXXZgr7M7t57+nD++1bO9s91z2Z2enpl+Px+PenRX1bervtU10+/+1re6ShGBmZkZQK7ZFTAzs5nDoWBmZiMcCmZmNsKhYGZmIxwKZmY2wqFgZmYjHApmVSS9T9Lnm10Ps2ZxKMwykuZLWi/ptVXTFkh6RNL5VdNWS7pe0jZJ2yXdJ+nDkpZk8y+WNCypNxsekvTHDa77WZI2NnIdk1GrPhHx1xHxRw1a33pJL2nEshthuvbXbHtf5jqHwiwTEb3AZcAnJHVnk/8WWBMRXweQ9FzgJuAnwFMjYjFwNlACnl61uJ9GxPyImA+cD/ytpGdMz5bYZEgqNLsO1iIiwsMsHICrgKuBs4AngRVV834M/NM4r78Y+PGoaT8DXls1fi5wL7CdFDInVc07KZu2PStzbtW8c4D7gF3Ao8C7gHlAH1AGerPh8Drb9WngP7LX3wo8ZQLvx1OB7wNbgQeACw6kPsCfA1/KXnc0EMAlwAZgG/Bm4NnAXdm2f6pqPU8BfpDtjyeALwOLs3n/mq2rL1vXuyfwHq8H/ixb1wBQyMYfzbblAeDFNd6LM4DNQL5q2u8Dd2XPnwOsAXYCjwMfq/OengVsrDNvEfBFoAd4GHg/kMvm5YF/yN6DdcBbs/exUGdZ64GX1Jn3JmBttl+vq/zNAAL+EdgC7Mjeo1Pq7e9m/7/OpqHpFfBwgDsOlgCbsn+8S6qmzwOGgbPGef3FVIVC9kG3HTghGz8B2A28FCgC787+Oduy8bXA+7LxF2X/gCdmr90EPL+qns/Mntf9kKmqx1XZB8Bzsg/BLwPXjPOaeaQP7Uuy1zwze1+eNtn6UDsUPgt0AL8N9APfBJYBK7MPpRdm5Y/L3q92oBu4Gfh41bL3+fAb6z2uKn8ncATQCZyYbefhVfWrGZjAg8BLq8a/Brwne/5T4PXZ8/nAGXWWUXd/kQLhW8CCrB6/Ai7N5r2Z9KG8Knu//5MDCIXs7+qJbH+2A/8E3JzN+x3gdmAxKSBOIvtiVG9/e5jY4MNHs1REbCN9w+wCrq2atYR0WHBzZYKkv836FXZLen9V2TOy6b2kVsK/Ar/O5r0a+I+I+H5EDAF/T/pgei7pm+h84CMRMRgRPwCuBy7MXjsEnCxpYURsi4ifT3Lzro2In0VEiRQKp41T/uXA+oi4MiJK2fq+QTokNhX1+cuI6I+I/0v6EL86IrZExKPAfwHPAIiItdn7NRARPcDHgBeOsdyx3uOKT0bEhojoI4V9e7YtxYhYHxEP1ln21WT7Q9IC0rfnq6vej+MkLY2I3oi4ZTJvhqR8Vvf3RsSuiFhPahm8PityAfCJiNiY/Z1+ZDLLr/I64IqI+HlEDADvBc6UdHS2DQtILURFxP0Rsalq+w5mf7c0h8IsJeki0je0/wQ+WjVrG+kwxYrKhIh4d6R+hX8nfZOuuCUiFkfqU1gOPA3462ze4aTDApVllEnfUldm8zZk0yoezuYBvIr0IfSwpB9JOnOSm7e56vkeUgCN5Sjg9CzgtkvaTvpAWT5F9Xm86nlfjfH5AJKWSbpG0qOSdgJfApaOsdyx3uOKDVXz1wLvILVmtmTrOrzOsr8CvFJSO/BK4OcRUVnXpaRWyi8l3Sbp5WPUsZalpBbiw1XTqvf/4dX1HvV8Mka/P72kQ3Mrsy8inyIdanxc0uWSFmZFD3Z/tzSHwiwkaRnpeOqbgP8BXCDpBQARsZt0HP6Vk1lmRDxO+nb9e9mkx0gftpV1inQY49Fs3hGSqv9+jszmERG3RcR5pEMs3wS+WlnNZOo0CRuAH2UBVxnmR8QfT3N9/iZb5qkRsRC4iHRoo2L0+sZ6j2u+JiK+EhG/mb0u2PcLQXW5+0gfqC8DXksKicq8X0fEhaT346PA1yXNm/hm8gTp2/hRVdNG9j/p8M2qqnlHTGLZ1Ua/P/OAQ9n7d/bJiHgW6cvMCcD/yqbX2982AQ6F2elTwDcj4odZk/ndwOeyb4Vk438o6T1ZgCBpFXBMvQVKOpTUGXlvNumrwO9KerGkIvBOUmfnf5NCZzfwbklFSWeRwuQaSW2SXidpUXZIZCfpsAekb9iHSlo0Re9DxfXACZJen9WnKOnZkk6a5vosIHUib5e0kuxDqsrjwLFV42O9x/uRdKKkF2X7uZ/UShmuVTbzFeBtwAtIfQqV5VwkqTtrmWzPJtddjqSO6oHUEv0q8OHsdOijgD8ltYwq2/V2SSslLSZ1jo+nOGo9haz+l0g6LdvmvwZujYj12f49PXvfdmfvx/A4+9smotmdGh4mNwCvIH2DWjxq+o3Ah6vGTwe+Q/qn3w7cA3wYODSbfzHpn6Vy5s0W0jHnZVXL+H1Sh+EO4EdkHbfZvKdl03ZkZX4/m94GfI90GGsncBvwm1Wvu4J0CGA79c8++quq8bMYp3M6K3ci6Yylnmz5PyD1RUyqPtTuaC5Uld9IVSc+6YPw/VXvye3Z+3kn6UN+Y1XZ84BHsnW9awLv8Xr27Zg+ldT3s4vUGX99rfewqvyRpA/w/xg1/UvZ/u4lfQl4RZ3Xn5Vt/+jhOFLf1Zey93sD8AH2nn1UILVknySdffQnpJaF6qxnfY11/FU2782kTvPK9q7Kpr+YdMZRL3vP9Jo/3v72MP6g7A02M2sISS8DPhsRR41b2JrOh4/MbEpJ6pR0jqRCdhjtg6STHGwWcEvBZgVJzwe+W2tepLOnbIaQ1EU6FPZUUr/HfwBvj4idTa2YTYhDwczMRvjwkZmZjZh1F9launRpHH300c2uhpnZrHL77bc/ERHd45WbdaFw9NFHs2bNmmZXw8xsVpH08PilGnj4SNIRkn4o6X5J90p6e40ykvRJSWsl3SXpmY2qj5mZja+RLYUS8M6I+Hl2Qa7bJX0/0s/vK14GHJ8NpwOfyR7NzKwJGtZSiIhNkV2dMCJ2Afez74W+IP3C84uR3AIslrQCMzNrimk5+yi71O0zSNfMqbaSfa+guJH9gwNJl0laI2lNT09Po6ppZtbyGh4KkuaTrr75jho/XlGNl+z3w4mIuDwiVkfE6u7ucTvPzczsADU0FLIrGH4D+HJEXFujyEb2vazuKtLF3szMrAkaefaRgC8A90fEx+oUuw54Q3YW0hnAjth79yQzM5tmjTz76Hmk2/PdLenObNr7SJfzJSI+S7q08zmk+9LuId1jtyG2PtnDhnUPcPzJp9HV5UvlmJnV0rBQiIgfU7vPoLpMAG9pVB2q3f5PF/FSbmH9vB9w9EnPmo5VmpnNOi1z7aOlZ74OgOFSqck1MTObuVomFJRLjaLy8FCTa2JmNnO1TigUslAoDTa5JmZmM1fLhEIun0Ihhn34yMysnpYJBeWLAAw7FMzM6mqZUMhloRDuUzAzq6uFQqHS0eyWgplZPS0UCllLoeyWgplZPS0UCpWzj9xSMDOrp2VCIZ+dkhplh4KZWT0tEwqVw0fuUzAzq691QqFQOfvIoWBmVk/LhEI+71AwMxtP64RCMbsgrPsUzMzqap1QcEvBzGxcjbzz2hWStki6p878RZK+LekXku6V1LAb7ADksz4F/DsFM7O6GtlSuAo4e4z5bwHui4inA2cB/yCprVGVGelo9uEjM7O6GhYKEXEzsHWsIsCC7F7O87OyDfvELoy0FBwKZmb1NPIezeP5FHAd8BiwAHh1RJQbtbK8T0k1MxtXMzuafwe4EzgcOA34lKSFtQpKukzSGklrenp6DmhlbimYmY2vmaFwCXBtJGuBdcBTaxWMiMsjYnVErO7u7j6gleXzlVNShw+stmZmLaCZofAI8GIASYcBJwIPNWplyuUoRc4tBTOzMTSsT0HS1aSzipZK2gh8ECgCRMRngb8ErpJ0NyDgzyLiiUbVB2CYvEPBzGwMDQuFiLhwnPmPAb/dqPXXMoxbCmZmY2mZXzQDlJRH7lMwM6urpUJhmDyEWwpmZvW0VCiUcUvBzGwsLRUKw+SQWwpmZnW1Vigoj9zRbGZWV0uFQpk8Ch8+MjOrp6VCwS0FM7OxtVYoUHBLwcxsDC0VCmXl3dFsZjaGlguFnFsKZmZ1tVYouKPZzGxMrRUKypPzj9fMzOpqvVBwS8HMrK6WCoVQnpw7ms3M6mqpUCirQN6hYGZWV2uFQq5IPoaaXQ0zsxmrYaEg6QpJWyTdM0aZsyTdKeleST9qVF0qhnNt5HFLwcysnka2FK4Czq43U9Ji4J+BcyPiacAfNLAuAESuSMEtBTOzuhoWChFxM7B1jCKvBa6NiEey8lsaVZeROuXaKLhPwcysrmb2KZwALJF0k6TbJb2hXkFJl0laI2lNT0/PAa+wnC9S8OEjM7O6mhkKBeBZwO8CvwP8H0kn1CoYEZdHxOqIWN3d3X3ga8wXKbqlYGZWV6GJ694IPBERu4Hdkm4Gng78qlErjFwbRdynYGZWTzNbCt8Cni+pIKkLOB24v6FrzBcp+vCRmVldDWspSLoaOAtYKmkj8EGgCBARn42I+yV9D7gLKAOfj4i6p69OiXwbBZUZLpXIF5rZSDIzm5ka9skYERdOoMzfAX/XqDrsJ98OwNDQgEPBzKyGlvpFM4UiAIODA02uiJnZzNRSoaB8GwBDA/1NromZ2czUUqHw44d2AFAackvBzKyWlgqFF5y8EoDS4GCTa2JmNjO1VCjkCqmjuTTkw0dmZrW0WCikjubhQYeCmVktLRYKqaUwXPLhIzOzWlosFNLZR8PuaDYzq6m1QqHYAfjsIzOzeloqFPLF1FIoD/nwkZlZLS0WClmfglsKZmY1tVQoFLJQCHc0m5nV1FKhMNJScCiYmdXUUqGwt6Xgw0dmZrW0VCgU29PZR+WSf7xmZlZLw0JB0hWStkga88Y5kp4taVjS+Y2qS0V7excA4Utnm5nV1MiWwlXA2WMVkJQHPgrc0MB6jGjrzEJhqG86VmdmNus0LBQi4mZg6zjF/ifwDWBLo+pRrb1zXnpSciiYmdXStD4FSSuB3wc+O23rzOUZjAK4T8HMrKZmdjR/HPiziBger6CkyyStkbSmp6fnoFY6oDbkUDAzq6mZd69fDVwjCWApcI6kUkR8c3TBiLgcuBxg9erVcTArHaCNnEPBzKympoVCRBxTeS7pKuD6WoEw1QbVTm7YoWBmVkvDQkHS1cBZwFJJG4EPAkWAiJi2foTRhtTmUDAzq6NhoRARF06i7MWNqsdoQ7l28sP+nYKZWS0t9YtmgJLaKJQdCmZmtbReKOQ7HApmZnW0XCgM59opOhTMzGpqvVDId1AMh4KZWS0tFwrlfDtt4fspmJnVMm4oSHqKpPbs+VmS3iZpceOr1hjlQgdtuKVgZlbLRFoK3wCGJR0HfAE4BvhKQ2vVQJHvoD2Gml0NM7MZaSKhUI6IEunidR+PiD8BVjS2Wg1U6KCdQSIO6moZZmZz0kRCYUjShcAbgeuzacXGVamx7np8kILKDAz4EJKZ2WgTCYVLgDOBD0fEOknHAF9qbLUa59RjlgPQ37e7yTUxM5t5xr3MRUTcB7wNQNISYEFEfKTRFWsUFdN9mgf7dsOSQ5tcGzOzmWUiZx/dJGmhpEOAXwBXSvpY46vWGLliuiXnYP+eJtfEzGzmmcjho0URsRN4JXBlRDwLeEljq9U4ubZOAAYHfPjIzGy0iYRCQdIK4AL2djTPWvn2FApDbimYme1nIqHwIeAG4MGIuE3SscCvG1utxilkLYWhAYeCmdloE+lo/hrwtarxh4BXNbJSjVTIWgrDDgUzs/1MpKN5laR/l7RF0uOSviFp1QRed0X2mnvqzH+dpLuy4b8lPf1ANmCyih3zASjd9v9Px+rMzGaViRw+uhK4DjgcWAl8O5s2nquAs8eYvw54YUScCvwlcPkElnnQOroWAlA6fqyqmZm1pomEQndEXBkRpWy4Cuge70URcTOwdYz5/x0R27LRW4BxWx9ToXP+IgCG+3dNx+rMzGaViYTCE5IukpTPhouAJ6e4HpcC3603U9JlktZIWtPT03NQK+pckEIhBhwKZmajTSQU/pB0OupmYBNwPunSF1NC0m+RQuHP6pWJiMsjYnVErO7uHreRMqb27PARA70HtRwzs7lo3FCIiEci4tyI6I6IZRHxCtIP2Q6apFOBzwPnRcRUtz5qrzOXZw/taNChYGY22oHeee1PD3bFko4ErgVeHxG/OtjlTcYeusgNORTMzEYb93cKdWjcAtLVwFnAUkkbgQ+SXXI7Ij4LfAA4FPhnSQCliFh9gPWZlP5cJ/mSL3NhZjbagYbCuHeoiYgLx5n/R8AfHeD6D8pArotiyT9eMzMbrW4oSNpF7Q9/AZ0Nq9E0GMx3URx2S8HMbLS6oRARC6azItNpKN9F19Djza6GmdmMc6AdzbNaqTCfjnJfs6thZjbjtGQolIvz6MShYGY2WmuGQts8uqKPiHH7y83MWkpLhgJtC+jUIP0Dg82uiZnZjDKRS2fvkrRz1LAhu5z2sdNRyamm7PLZu3ftaHJNzMxmlon8TuFjwGPAV0ino74GWA48AFxB+oHarJJrTydW9fVuh+5lTa6NmdnMMZHDR2dHxL9ExK6I2BkRlwPnRMS/AUsaXL+GyHemi+L173ZLwcys2kRCoSzpAkm5bLigat6s7Kltu//fAejvdSiYmVWbSCi8Dng9sCUbXg9cJKkTeGsD69Ywnb/1TgAGeuveA8jMrCWN26cQEQ8Bv1dn9o+ntjrTo2vRUgAGd03L1brNzGaNiZx9tCo702iLpMclfUPStNw6s1EWLEk36hne7ZaCmVm1iRw+uhK4DjgcWAl8O5s2a3UuPBSA6Ns2Tkkzs9YykVDojogrI6KUDVcBB3dPzCZTvkgvXeT6tze7KmZmM8pEQuEJSRdJymfDRcCsPxjfq/kUBhwKZmbVJhIKfwhcAGwGNgHnA5eM9yJJV2T9EPfUmS9Jn5S0VtJdkp45mYofrD35hbQN+ZRUM7Nq44ZCRDwSEedGRHdELIuIVwCvnMCyrwLOHmP+y4Djs+Ey4DMTWOaU6S8upKO0czpXaWY24x3oBfH+dLwCEXEzMNbpPecBX4zkFmCxpBUHWJ9JGyouomt413StzsxsVjjQUNAUrHslsKFqfGM2bf+VSZdJWiNpTU9PzxSsGoY7FrMgHApmZtUONBSm4vIWtYKl5nIj4vKIWB0Rq7u7p+bEp3LHISyil/7B0pQsz8xsLqj7i2ZJu6j9IS2gcwrWvRE4omp8FelqrNNCXUsoqMyT27fRsWxWn2FrZjZl6rYUImJBRCysMSyIiIlccns81wFvyM5COgPYERGbpmC5E9K24b8B2P7k5ulapZnZjNewO69Juhr4KXCipI2SLpX0Zklvzop8B3gIWAt8Dvj/GlWXWjqf+yYAerdOW+PEzGzGm4pv/DVFxIXjzA/gLY1a/3jmH5r6tPsdCmZmI1rzHs3A4sNSd8ZP77q/yTUxM5s5WjYUOhYdxjBi9aGDza6KmdmM0bKhQC7PDi0iv2dqfvdgZjYXtG4oAFtiEaUd03bCk5nZjNfSoRBdy1ie9/WPzMwqWjoUBju7WTS8lXQilJmZtXQo3LOjnaVsZ2ffULOrYmY2I7R0KJx28km0aZjNmzY2uypmZjNCS4fCvMOOBWDbY2ubXBMzs5mhpUNhyeHHAbB7y7om18TMbGZo6VBYuPwYAMpbH25yTczMZoaWDgV1LmZ7zGProz58ZGYGDbwg3myxs305R5efbHY1zMxmhJZuKQA8NHQoiwc3+7cKZmY4FFhW7OMI9bBlZ1+zq2Jm1nQNDQVJZ0t6QNJaSe+pMf9IST+UdIekuySd08j61NL+jAvo0gDv+sJ3pnvVZmYzTiPvvJYHPg28DDgZuFDSyaOKvR/4akQ8A3gN8M+Nqk89hxx1CgAXla6d7lWbmc04jWwpPAdYGxEPRcQgcA1w3qgyASzMni8Cpv02aIuPTKFwb/+y6V61mdmM08hQWAlsqBrfmE2r9ufARZI2ku7Z/D9rLUjSZZLWSFrT0zO19z/QvG520cVvlO+b0uWamc1GjQwF1Zg2+hSfC4GrImIVcA7wr5L2q1NEXB4RqyNidXd39xTXUjyoI1lU3uEzkMys5TUyFDYCR1SNr2L/w0OXAl8FiIifAh3A0gbWqab5q07hRD3Cw0/snu5Vm5nNKI0MhduA4yUdI6mN1JF83agyjwAvBpB0EikUpv3+mF1HP4tF2sOvf3XPdK/azGxGaVgoREQJeCtwA3A/6SyjeyV9SNK5WbF3Am+S9AvgauDiaMIxnO4TnwvATT+4YbpXbWY2ozT0MhcR8R1SB3L1tA9UPb8PeF4j6zARxeUnMxgFThu8vdlVMTNrqpb/RTMAhTaemH8iR2oLuwdKza6NmVnTOBQyccTpnKa1rHlwU7OrYmbWNA6FTPepL6VdQ2z4xU3NroqZWdM4FDJtxz6PMjli3c3NroqZWdM4FCo6FvHkoqfxG/1rWLult9m1MTNrCodClfZyP6flHuInt9/Z7KqYmTWFQ6HKwjd8BYAdd3zTl7wws5bkUKjWfQLrtYrVfT/hpw/6Fp1m1nocCqOs+s3Xckbufj785e82uypmZtPOoTBKYfXFlCV+b+gGfrl5Z7OrY2Y2rRwKoy1aSXQeyqvzP+SN//KjZtfGzGxaORRqKL7miyxRL783+F1++MCWZlfHzGzaOBRqOeq5lI/9Ld5S+BZvv/JH7OwfanaNzMymhUOhjtxL/pzF2s07C//GMz70fYbLPkXVzOY+h0I9h5+GTn8zbyx8n9VxH8/9mxv92wUzm/McCmN58f+BJcfw8fbPMrSrh9P/+ka3GMxsTmtoKEg6W9IDktZKek+dMhdIuk/SvZK+0sj6TFrbPDj/CyzXk3yj/S/YvquXE97/XZ7sHWh2zczMGqJhoSApD3waeBlwMnChpJNHlTkeeC/wvIh4GvCORtXngK18FnrVFzhGm/hex/solAdY/Vf/ybfufNSHk8xszmlkS+E5wNqIeCgiBoFrgPNGlXkT8OmI2AYQETPz/M9TXgkv/zjH8hg/P+YzdGs7b7/mTp7yvu9w/V2PURouN7uGZmZTopGhsBLYUDW+MZtW7QTgBEk/kXSLpLNrLUjSZZLWSFrT09PToOqOY/Ul8MrPMW/zbdza+XaueP4uygFv/codHP+/v8uHvn0ft63fStl9DmY2ixUauGzVmDb6E7MAHA+cBawC/kvSKRGxfZ8XRVwOXA6wevXq5n3qnvoHsPwU9LkX8aLb/gcPPf9N/HjFJXzx7j1c8ZN1XPGTdQh41bNW8cwjl/DMoxZz/LIF5HO13gozs5mnkaGwETiianwV8FiNMrdExBCwTtIDpJC4rYH1OjjLToL/9SB8/wPkbvscL9AXeMGZb2X3yy7lxk1tfO+eTXzj9o18/faNIy85/ZhDOP6w+RzXPZ/jli3guGXzOWxhO5LDwsxmFjWqs1RSAfgV8GLgUdIH/Wsj4t6qMmcDF0bEGyUtBe4ATouIutetXr16daxZs6YhdZ60J9bCTX8D93w9jZ94DpzyKuKEs3l4l7hjwzbueGQ7dz+6gzse2b7fy488pIvlizpYsagjPS7sYMXizpHxpfPaybmVYWZTQNLtEbF63HKNPING0jnAx4E8cEVEfFjSh4A1EXGd0lflfwDOBoaBD0fENWMtc0aFQsX2DXDb5+EnnwAClIOTzoWnvQKOPQs6lxAR9PQOsHZLLw9u6WXj9j427+hn045+frZua91Fr6wKieULOzhkfhtLutpY0lVkUWcbS+YVWdLVxuKuIu2F/HRtsZnNMjMiFBphRoZCRXkYHrkF7r0WbvsCI10oy38Djn4BHHUmrHg6LDoCRh06KpeDrXsGR4Ji846+7LGfx3b0cccj2xkojX+W08rFnSzu2hsUlQBZ3JUCZHFX2z7TFnYUfBjLrAU4FJqtPAwbfgbr/wvW3ZweKzqXpKBY8XRY/nRYcSocehzkxv+m3zc4zLY9g2zbM8j2PUPZ8yG2784e9wyyvW+IH/xycmf3Hts9byQsFnYUWdBRYGFnelxQGe/Yd3x+e4GutrxDxWwWcCjMNEP98Pg9sOlO2HQXbL4LHruTkdZEsQsOOyUFxPJT0+PSE9Kvqg/ScDnY2VcVIPs8pnCpDphfbt7JZP8s2vI5jlk6j/lZWMzvKDC/LT3Oay+woL3G87bCSLjMay/QVvBVV8waZaKh0Mizj6xasQNWrU5DxfAQ9DyQAqISFGuuhBjeW2b+YXDIsdlwTHpckj12Lp7QqvM5sWReG0vmtU2qyhHB7sFhdvUPsau/xM6+7LF/iN0DaXrvQIld/SV6B0r09pfYNTDE9r4hNm7bw0M9u/c7B3kiJHjq8oXMb88zr31vkFSeV6bPb0/B0tWep6utwLy2PF3tBbqKeTrb8rQXcm7FmE2SWwozTbkM29bB5rth64Ow9SHYui497tq0b9nOQ6oCY1RwdB26X79Fs5TLwe7BErsHhukdGKJ3YJje/lLV8xQuvZX5/en57oESuwdT4DzUs/uA1y/giEO66GpLYdHVlqezWMge905L8wt0FvOjylaep9d0ZPOLebdsbPZwS2G2yuXg0KekYbTB3bBt/b5BsfWh1Ll991f3Ldu+cG9AVIZKC2PB8mkNjFxOWT9EEeg4qGWVy8GeoRQYvQMldg+U2DM4zJ7B7HEgez40TN/gcDZvmL5sft9QGt+6u4++wRLrn9xz0Nsn4Oil8/YJmI5ifiRcKiFSmd+ZPa+U6Sjm6Sjm0muLBTracnRl4eQfPtp0cyjMJm3z4LCnpWG00gBseziFxLaqwNj0C7j33/ctW+yCJUens6AWHwGLVqXnlccFyyfU6d0MuZxSn0V7gcN7M/S5AAAN7klEQVSmcLkRwdBwpCAZygKkKkT6BktVz4dHAqavKpD6h9K03oESPbsGWPfE7gmdMTZRRx3atW+IVAdLW56OQp7OtlxVmarwacvRUUjl9g2kHB1FH2qzvRwKc0WhHbpPSMNow0OwY0NVC2NdCo4dG2DDrdA/6od1ysPClVlIrNz7fOHKbHwVdB0yYw5PTQVJtBVEWyHHIooNWcfQcHkkVPqHhukv7Q2e/qFh+ofKVWFTom+wzJ6hEgND5fSarHx/qUz/4DBP9A7SPzTMr7f0Tnldjzq0i/ZCCoyOQp72LDzSeG4kmCrT2gs52rPHSpn2qrLtxdze5RT2vsY/zpx5HAqtIF/cewiploFdsONR2LExBcWOjXufb/gZ7HwMyqPuU13ohIWH7zssOBwWrtj7fF435P0nVlHM51jUmWNRZ2NCB/a2ePpLw/QPpqDpG9obPH1D2fTSMH2D5ZHpA0PDDJTKI+HUX9obVP1Dw+zsT62fX27e1ZB6F/PisIUde0OlOmCKOdoLedryOdqLuRRAhTS/rZDbJ5DaqubtM17M7Te/Le9QqsX/sQbtC2DZU9NQS7kMu3tg58YUHjuzANn5KOzcBI/8ND2ODg7l0tlTC5bDghVpWLgiBcaC5Vl4rICORXOq1dFM1S2ehR2NC5+KiGBwuDwSKANDZQZKKUwqj/2jAqfe/H2CqZSWtbN/iP6hMoOlVP7xnY25wZWA5Ys6RgKnbZ9QqTWtVpnxy42eNxPDyaFg48vlYMFhaVj5rNplymXY80RqVex8LJ0ptWtTCotdm1J/xyM/hb5t+7+22JUFRyUsagTHguXpEJnNKJKyD7f8tIRQtVIWRikwUmhUPx8YKjMwXB4JquqylZDZ5/lQOQVc5TGb1jtQGlnGYKnMll0DlBp0ifxCTiyd375f4KRAyfPyU1dwweojxl/QwdShoUu31pHLwfxlaTj8tPrlhvqywNhcFR5VzzfelsaHa3wj7DoU5i2DeUvTeuZ1p+fzKs+7986bgh/92cxWyOco5HPMa+J3hXJ5b0tpcFQwjQ6rwdLe0ElhNczg8P5BNbhfeKXHHX1D9A0Oj1+pg+RQsOlV7By7fwMgIrUoarU4dvek4bE7oLcHBusc4y52jQqMpemxVph0HTJjz7aymS2XEx251AcyVzgUbOaR0gd11yGw/JSxyw71we4n9obFyPAE9G5Jz3dsTCGyu2ffX4vvXWFqhcxftjc8ag3zs0e3QmwOcyjY7FbsTL+1WDyB46zlcjr9thIcvVuqAqXq+YRaId37H7KqHncrxGYph4K1jlxubwuk+8Txy0+4FfLzNK1WK0S5rC+k6hDWvOoWydJ01dzOrF6dS9IpxGZN0tBQyO6s9gnSTXY+HxEfqVPufOBrwLMjYg5f2MhmlQNthVQC40BaIZAuUdK5JAuJqrAYeX4IdI0ab1/g03ptSjQsFCTlgU8DLyXdi/k2SddFxH2jyi0A3gbc2qi6mDXcAbVCemDPk7Bna+pY79uWPd+67+PWB2HPNhjYMcb6Cykc9gmT0UFSI2QKk7tyrs19jWwpPAdYGxEPAUi6BjgPuG9Uub8E/hZ4VwPrYjazFDth8ZFpmKjhUmqN1AqO6se+7enCiY/9PJ3eO9YFzNvmjx0gtULGPzac0xoZCiuBDVXjG4HTqwtIegZwRERcL8mhYDaWfCHri1g68ddEwNCeGsGxLbU+RofK9ofT4+jrYVVTft+wGHk++pDXqHApHtwVcm16NDIUan2VGPnKIikH/CNw8bgLki4DLgM48shJfLMya3VSOoW2bd7E+kYqysPQv2MCrZJtqbN9811pWqmv/jKLXfu3SjoWp5bHPkONaQ6UadPIUNgIVP8VrgIeqxpfAJwC3JRdsnc5cJ2kc0d3NkfE5cDlkG6y08A6mxmk02grfSSTMdRXp1Wydf8+k813p+Dp3wHDg2MvN99eIzxGDwvrhMwiKHT4kNcENTIUbgOOl3QM8CjwGuC1lZkRsQMYaQdLugl4l88+MpvFip3p8uqLVk7udUP9ewNiZNheY1rVsP2RveXGDZW2CYRKnVZKi4VKw0IhIkqS3grcQDol9YqIuFfSh4A1EXFdo9ZtZrNMsSMNCw7w1kkHFCobHCo1+B7NZmaTCZWBnfuO7+6BGOcOe+OFSvvCsUOl2HnQoeJ7NJuZTdRUtFT2CYtxWin9O1IHfeV5qX/s5eeKKRzO+GN4QWNP1HQomJkdrEqozF92YK+faKgsPX5q612DQ8HMrNkONlSmUK7ZFTAzs5nDoWBmZiMcCmZmNsKhYGZmIxwKZmY2wqFgZmYjHApmZjbCoWBmZiNm3bWPJPUADx/gy5cCT0xhdWYDb3Nr8Da3hoPZ5qMionu8QrMuFA6GpDUTuSDUXOJtbg3e5tYwHdvsw0dmZjbCoWBmZiNaLRQub3YFmsDb3Bq8za2h4dvcUn0KZmY2tlZrKZiZ2RgcCmZmNqJlQkHS2ZIekLRW0nuaXZ+pIukIST+UdL+keyW9PZt+iKTvS/p19rgkmy5Jn8zeh7skPbO5W3BgJOUl3SHp+mz8GEm3Ztv7b5Lasunt2fjabP7Rzaz3wZC0WNLXJf0y299nzuX9LOlPsr/peyRdLaljLu5nSVdI2iLpnqppk96vkt6Ylf+1pDceaH1aIhQk5YFPAy8DTgYulHRyc2s1ZUrAOyPiJOAM4C3Ztr0HuDEijgduzMYhvQfHZ8NlwGemv8pT4u3A/VXjHwX+MdvebcCl2fRLgW0RcRzwj1m52eoTwPci4qnA00nbPyf3s6SVwNuA1RFxCpAHXsPc3M9XAWePmjap/SrpEOCDwOnAc4APVoJk0iJizg/AmcANVePvBd7b7Ho1aFu/BbwUeABYkU1bATyQPf8X4MKq8iPlZssArMr+UV4EXA+I9CvPwuj9DdwAnJk9L2Tl1OxtOIBtXgisG133ubqfgZXABuCQbL9dD/zOXN3PwNHAPQe6X4ELgX+pmr5PuckMLdFSYO8fWMXGbNqckjWZnwHcChwWEZsAssfKzV/nwnvxceDdQDkbPxTYHhGlbLx6m0a2N5u/Iys/2xwL9ABXZofNPi9pHnN0P0fEo8DfA48Am0j77Xbm/n6umOx+nbL93SqhoBrT5tS5uJLmA98A3hERO8cqWmParHkvJL0c2BIRt1dPrlE0JjBvNikAzwQ+ExHPAHaz95BCLbN6u7NDH+cBxwCHA/NIh05Gm2v7eTz1tnPKtr9VQmEjcETV+CrgsSbVZcpJKpIC4csRcW02+XFJK7L5K4At2fTZ/l48DzhX0nrgGtIhpI8DiyUVsjLV2zSyvdn8RcDW6azwFNkIbIyIW7Pxr5NCYq7u55cA6yKiJyKGgGuB5zL393PFZPfrlO3vVgmF24DjszMX2kgdVtc1uU5TQpKALwD3R8THqmZdB1TOQHgjqa+hMv0N2VkMZwA7Ks3U2SAi3hsRqyLiaNJ+/EFEvA74IXB+Vmz09lbeh/Oz8rPuG2REbAY2SDoxm/Ri4D7m6H4mHTY6Q1JX9jde2d45vZ+rTHa/3gD8tqQlWSvrt7Npk9fsDpZp7Mg5B/gV8CDwv5tdnyncrt8kNRPvAu7MhnNIx1NvBH6dPR6SlRfpTKwHgbtJZ3c0fTsOcNvPAq7Pnh8L/AxYC3wNaM+md2Tja7P5xza73gexvacBa7J9/U1gyVzez8BfAL8E7gH+FWifi/sZuJrUbzJE+sZ/6YHsV+APs+1fC1xyoPXxZS7MzGxEqxw+MjOzCXAomJnZCIeCmZmNcCiYmdkIh4KZmY1wKJiZ2QiHgtkESDpN0jlV4+dqii7BLukdkrqmYllmB8u/UzCbAEkXk34o9NYGLHt9tuwnJvGafEQMT3VdzNxSsDlF0tHZDWg+l92g5f9K6qxT9imSvifpdkn/Jemp2fQ/yG7s8gtJN2eXRvkQ8GpJd0p6taSLJX0qK3+VpM8o3ezoIUkvzG6ccr+kq6rW9xlJa7J6/UU27W2kC779UNIPs2kXSro7q8NHq17fK+lDkm4FzpT0EUn3ZTdb+fvGvKPWcpr9E28PHqZyIF2XvgSclo1/FbioTtkbgeOz56eTrpcD6fIBK7Pni7PHi4FPVb12ZJx0k5RrSJcgOA/YCfwG6UvX7VV1qVyqIA/cBJyaja8HlmbPDydd96ebdGXUHwCvyOYFcEFlWaRr6au6nh48HOzgloLNResi4s7s+e2koNhHdqnx5wJfk3Qn6aYkK7LZPwGukvQm0gf4RHw7IoIUKI9HxN0RUQburVr/BZJ+DtwBPI10F8DRng3cFOnqoCXgy8ALsnnDpKvhQgqefuDzkl4J7JlgPc3GVBi/iNmsM1D1fBiodfgoR7phy2mjZ0TEmyWdDvwucKek/cqMsc7yqPWXgYKkY4B3Ac+OiG3ZYaWOGsupdV38iv7I+hEioiTpOaSrh74GeCvpMuJmB8UtBWtJkW5EtE7SH8DIDdGfnj1/SkTcGhEfIN3W8QhgF7DgIFa5kHRjnB2SDmPfG8ZUL/tW4IWSlmb3Fr8Q+NHohWUtnUUR8R3gHaQrqJodNLcUrJW9DviMpPcDRVK/wC+Av5N0POlb+43ZtEeA92SHmv5msiuKiF9IuoN0OOkh0iGqisuB70raFBG/Jem9pPsGCPhORHxr/yWyAPiWpI6s3J9Mtk5mtfiUVDMzG+HDR2ZmNsKHj2zOk/Rp0r2dq30iIq5sRn3MZjIfPjIzsxE+fGRmZiMcCmZmNsKhYGZmIxwKZmY24v8B90b8DxRosfIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2418084ceb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "        \n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(0, cvresult.shape[0])\n",
    "        \n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators4_1.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1oAAANGCAYAAADktv9+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xu4ZGdZJ+zfm8Sd0N10c+hwaAgEphGJyDEooAKa4YsaPIzjIAhKhoMjfCoMRgxMvhCYEOLYcnaGYUROMooDikLEIBGUgQCGkyDhECQhIRDSBLrTaTot8H5/rCq6enftvWtXr9qrDvd9XXVl16pVa7+1dwf6l+dZz1tqrQEAAKA9x3S9AAAAgHkjaAEAALRM0AIAAGiZoAUAANAyQQsAAKBlghYAAEDLBC0AAICWCVoAAAAtE7QAAABaJmgBC6OU8kellJtLKT8w5LWzSym1lPLTy45v7b32wVLKN0op/1pKua6U8jellF8qpRw/cO7JvWsMPvaWUj5eSnlGKeXYjficqymlPK2UcmbX6zgapZRNpZTzSimPGPLamb2f+8kdrGtHb1332+jv3aVSymtLKfu6XkeS9H7+tZSyveu1AAhawCJ5RpKvJHldKeV7+gd7wet5SV5ba33bwPF7JPlokv+S5P8m+ZUkP57kN5J8KckfJTlnyPd5eZKH9B6PTvK+JC9O8t/a/0jr9rQkZ3a9iKO0KclzkzxiyGsXpfm5f3kjF9SzI826FipoATDccV0vAGCj1Fr3llKelOSdaQLSc3uB6w1JrksTxJIkpZTjkrw1yW2S/GCt9fJll/uzUsrzk9x/yLf6Yq31AwPP/6aUcu8kj03yW619II5Qa70+yfVdr6NNpZRNtdb9Xa8DgPVR0QIWSq31XUlemeQ5pZQHJjkvyX2TPKnWumfg1H+X5JQkLxgSsvrXuqrW+tYRv/WeJP86eKCUckwp5VmllE/3Whq/Wkp5fSnlzsvfXEp5Yq8F8UAp5YZSyl+UUu617Jy7l1L+tJRybe9615VSLum3spVSrkzy/UkePtDaeOVqi+6d84pSyi+XUi4vpezvreNRI37uwWstlVLOGfi815dSXlNKOXHZeT9eSnlPKeVrpZRvllK+WEp5S69l8OQcClLPHfgcr+2994jWwd61PllKeUgp5f29a15ZSvmPvdfPKKV8pPfZPlFK+Yll69nZW+fneud8qZTytsEW1F4b4z/2nr5mYF3nDZzzM6WUS3vXuLGU8rellIcs+1791rcHlFLeXEr5epLP915b9fe7ws/8Gb3r7Rzy2u+WUg722+xKKfcvpby99+fw5t73uWjYn8dxlFJ+pLfeG3s/g/eXUs5Y4bxLe3/Wv1RK+a+llCcv/70e5VruXUr5y1LK13vf52OllCcsO+eY3p/Xz/T+zHyjlPJPpZSnD5xzYinlVaWUqwf+TL+vlPJv21gnMNtUtIBF9NtJTk/y5iQnJXllrfVvl53zyN4//2qM6x9TmopYkmxL8rNJfiLJ7y47738k+dUkr0jy9iQnJ/mvSR5RSnlArXV3kpRSnp3kgiR/kuTZSW6bJiBeWkp5UK31c73r/XWSY5M8K8kXk2xP8tAkt+q9/u96n3lPmhbCJLl5hM9zRpIHJTk3yb7e9f+ilHLPWuu/jPD+lFKOSfKXSX40TQvl+5PcNU3L5ntKKafWWr/Z+4v0RUnem+SJSb6R5E5pfn5LaVoCfyLJ3yR5dZI/7H2LtapYd0jymt73viZN++cflVJOSvILaX6+e3qf8a2llLvXWq/tvXdHkq8lObv3fW6T5AlJPlhKuX+t9TNJPpLkP/a+x/m9z5De90op5ZeSvDFNNfWxSY7v/RzfU0o5rdb6f5et98+T/Gma/yiwuXdsrd/vMH+c5s/dmRlocy3N/YKPT/K2WuvuUsrmJH+b5AtJ/t80Fd47JPmxJLdc5fojKaU8vHf9f0rypDR/7p6W5G2llMfWWt/UO+8+vfM+m+ZnvD/Jr/XW2opSyj3T/Pn7apLfTPO7fXyS15ZSbl9r7bf4PivNv2fnJ/mHJN+T5Pty+M/7DUkekKa9+LO91x6Q5t9RYNHVWj08PDwW7pHmL7s1zV/ctwx5/R29149fdryk+Y9U/cexA6+d3HvPsMdrlp37fb3jf7Ds+j/YO/6C3vNbpfnL5kXLzjspyYEkb+w9v23vfU9f43N/Msl71vFzqmnua7vlwLHbJ/l2krPXcZ3H9K7188uOn9o7/tTe83/fe37fVa61vXfOeUNeO7P32skDx97TO/bAgWO3SfKt3s92x8Dx+/bO/Y1Vvv+xaf7S/dkkLxryWc5cdv4xae7p+6ckxwwc35Im0Lxv4Nh5vWs8b9k1Rvr9rrDetyS5etn3/sne9R7Ve/7A3vOfHeP6r02yb41zLu191i0Dx45N8one2krv2J+lCfPbl/38/nn573WF79P/+W1f5Zw/6f27c9Ky43+d5KYk23rP35bko2t8vxuTvHi9PzMPD4/FeGgdBBZOr7ryG0m+k+R2af5yPaqnp2kB7D8+PuScl6apAD0oTUXgOWmGYvzJwDk/1vvnawffWGv9UJLLk5zWO/SQJLcYct7VSf5u4Lwb0rSY/XYp5Zm9NrC2/jf+3bXWGwe+93VpqgF3Xcc1HpWmOvW2Uspx/UeSj6UJco/onfexJAeTvKqU8oRSyt3b+ABJvlxr/XD/Sa31hjSf4WP1UOUqaX72ycBn6631OaWUT5VSDqYJaAeT3CPJYe2bK7hnmqrYG2qt3xlYw740IejBpZRNy97zlmXPj+b3+5okd04y2M72H9P83N/Re35Fkq8n+d1Syq+VUk4Z8dpr6lXLfijJm3ufOUlSa/12morQndP8jJLk4Un+rvaqub3zvpMmgLXlx5Nc0vt3aNBr0wxa6bdzfijJfUsp/72UcnopZeuQa30oyZm9FsMHl4EhOwCCFrCIzkrzl6lfSvK5NC1kt1h2zhd7/1weJv53DoWoj6xw/WtqrZf1Hu+ptb4wTUvgfyilnN47p99aNGw63rUDr490Xq21pgldF6dpefpIkutLKS8rpRxt69fXhhy7OU0AHNXt01TnDubwoPqvaVrUtidJrfXzaQLBV5P8QZLPl1I+P3hfzJhuGHLs4PLjtdaDvS9PGDj8ojS/v7cm+ek0oeFBaUL2KD+DtX6HxyS59bLjh517lL/fd/Su178n7dZJfibJ63thJ7W5P/HhaYLuBUn+uXeP1vNaCA+3TlMJXunzJ4f/eb9uyHnDjo3rtiOu5YVp/rfiwWl+hl/r3WN26sB7fjHJ65I8OU3V7obS3Gd5hxbXC8woQQtYKL3/Uv/8NH/JfFOaVrOdSV6w7NT+PVs/M3iw1vrVfohK0zY0qn/q/bNfPeuHlzsOOXdHkt3rPC+1Gc7xpFrrHdJUCF6c5j6Y31vHOidld5rP8qAVHv17xlJrfW+t9afT3N/24DR/gX1JKeUxG73onsen+fPynFrrxbXWD/V+/6Pu1bTW7/A7aapJg+ryE8f9/Q5Ujn6ulHKrNP+B4fg0la7B8z5Ra31MmqBxvyRvSnPP2tFOyvx6ms+40udPDv/zfvsh57UZXL42ylpqrd+qtb6o1vqANK2mj03TsntxvwJZa91da31GrfXkNP9R5tlJfj7LKtDAYhK0gIXRa1V7XZq/SD09SWozhv1FSZ5eSvnhgdP/Ismn0kwn/L4Wvn1/MtxXe//8u94/D7vJv5TyoDTtaJf0Dl2a5JtDzrtzei1Qw75ZrfWztdbz09wD84CBl9ZbiWrL29P8Bf7YgWrf4OMzy99Qa/12rfWDaYYzJIc+R3+Ax0Z9jpplQ0N60/LutOy8ldb1mTT3aP1SKaUMXGNzmnvSLq3rHN++yu93Ja9JU6V7bJr/uHBprfXTK1y71lo/Xmv9z2naPUe5/mprvSnJB5P8/GDluNf6+Pg0A0M+2zv890l+vAxsONw77z8czRqWuaT3PXYsO/4rae7Z+8DyN9Rav1FrfXOaKutt0tyPufycL9ZaX5HmP9Ic1c8MmA+mDgKL5NlpBhb8ZK31GwPH/780LWF/VEq5X631m7XWb5dSfi5Nq9aHSin/K81Qha+naYH7oTTVqWGj3+9SSnlw7+vNadoUn53kqjTT5FJr/Uwp5VVJfqOU8p00rUknp2lRuzpNtSK11m+UUv5rkgtKKa9Pc5/XbdNsjHsgzdS+/rS2VyT5P2naIQ+mCWL3SXLhwNo+keQxpZRfTPIvSQ7UWj+xnh/imP40yeOS/HUp5aVp7m351zT35/xYkr+stf5FKeXXeuu+KE375glppg8mybuSpNZ6YynlqiQ/W0q5JE373+5a65UTWvvb09yH8+k0lckHpplcec2y8z6fJhQ/rpRyeZqhDtfWWq8tpTwrzdTBt5dS/meaitJvp/mzdPZaC1jH73eoWuunSymXpvlzeFKaaZeD139UmurYW9P8uShpKjO3yqHq7mqOLaX8wpDjN9Va39H7vn+b5N2llF299T8tyb2TPLbXGpk0leWfTnJJKeUFaX6ev5ZDkxe/k9H8dCnliIpzLyw9L809g+8uzV54N6T5s3lGkmf12ihTSnlbmuExl6WZNnnXNHvtXZXkc6WUbUnenaad+NNpKtwPSjMV889HXCcwz7qexuHh4eGxEY80oehgklet8PqD00zSe9Gy41vT/CXxQzm0F9Z1acZ0Py3JpoFzT86R0wa/maai8eIkd1h27WPS3G/zmd7ark9vOMCQ9T0pzT1BN6epMrw1ySkDr98uTdWi/xf8G3vnPyOHTzu8a5rwuLe3vivX+LnVJK8YcvzKJK9d5+/guDRtaB/r/Vxu7K33lUl2Dvwe/rx3/QNpqo/vSfLTy651Wpr7lA701vja3vEzM3zq4CdX+AxvX+szpwkbf9j7vd+UZvT8j/Su+55l731M7zMdzLLJiGnG/H+g99n3pQmOD132/vMyZGreqL/fNX7+T+lde3+Srcteu2eawHBF7/VvpKlCPWGE6742K0/bvHLgvB9JU03a1/sel6Y39XDZ9X6k93M6kOZeqv+W5t+Tmt5EwFXWct4qa6kD5907zdYN30jz79THcuS0yGcmeV+afy9vThOw/jDJXXuvH59mi4aPp/nfhv1pAtd5GfjfBQ8Pj8V99MepAgBMpVLKO9OE5+/tei0Ao9I6CABMjVLKi5J8NE0L7W3StPU9Mk1VF2BmCFoAHJXekJHVfKcO7B8Fazg2zWTQO6Rp+ftUkl+utf5xp6sCWCetgwCMrZRycpIvrHHa82qt5018MQAwRVS0ADga16aZtLbWOQCwUFS0AAAAWmbDYgAAgJZpHRyilFKS7EizTwkAALDYbplmE/qR2wEFreF2JLmm60UAAABT485JvjTqyYLWcDcmydVXX52tW7d2vRYAAKAje/fuzUknnZSss9tN0FrF1q1bBS0AAGDdDMMAAABomaAFAADQMkELAACgZYIWAABAywQtAACAlglaAAAALRO0AAAAWiZoAQAAtEzQAgAAaJmgBQAA0DJBCwAAoGWCFgAAQMsELQAAgJYJWgAAAC0TtAAAAFomaAEAALRM0AIAAGiZoAUAANAyQQsAAKBlghYAAEDLBC0AAICWCVoAAAAtE7QAAABaJmgBAAC0TNACAABomaAFAADQMkELAACgZYIWAABAywQtAACAlglaAAAALRO0AAAAWiZoTbH9B7+Vk8++KCeffVH2H/xW18sBAABGJGjNiFPOvVjYAgCAGSFozRBhCwAAZoOgNWOELQAAmH6C1gwStgAAYLoJWgAAAC0TtKZYrXXF11S0AABgeglaU6yUsuJrp55/ibAFAABTStCaYe7VAgCA6SRozThhCwAApo+gNQcELQAAmC6C1hxwvxYAAEwXQWtOaCEEAIDpIWjNEWELAACmg6A1xTYtHZdPPf/0rpcBAACsk6A15dYbtlS1AACge4LWHBK0AACgW4LWDFhvVevU8y/JyWdfJHABAEBHBK0ZsWnpuFx2zmnreo82QgAA6IagNUM2LR237vcIWgAAsPEErRkyzhRCmxkDAMDGE7QWgBZCAADYWILWjLG3FgAATD9BawaNOxjDJEIAANgYgtaMGmcwRmI4BgAAbARBa0aN20JoOAYAAEyeoAUAANAyQWuGjVvVcr8WAABMlqA1445mCqGx7wAAMBmC1hwYZwphn6AFAADtE7TmxPYtJxiOAQAAU0LQmiPu2QIAgOkgaM0ZbYQAANA9QWsOjbuZsTZCAABoh6DFYUwiBACAoydozaGjGfmeCFsAAHC0BK05JWwBAEB3BK05tmnpuFx54RljD8cQtgAAYDyCFqsStgAAYP0ErQUw7hTCPvtsAQDA+ghaC+Bo79fqE7QAAGA0gtaCaCNs2WcLAABGI2gtkDbCljZCAABYm6C1YDYtHTf2FMJBghYAAKys1Fq7XsPUKaVsTbJnz5492bp1a9fLmYj9B7+VU869+Kiv86nnn37UwzYAAGBa7d27N9u2bUuSbbXWvaO+T0VrQbU1IMP4dwAAOJKgtcDaaiM85dyLs3vfgRZWBAAA80HQWnBttf2ZSAgAAIe4R2uIRbhHa7m27tlK3LcFAMD8cI8WR6Wte7YSEwkBAEDQ4rvaumfr1PMvsdcWAAALTdDiMG22/NncGACARSVocZg2Wwj7BC0AABaNYRhDLOIwjGHaHJCRGJIBAMDsMQyD1rV1z1afVkIAABaFoMWqtm85QSshAACsk6DFmtq+b8tUQgAA5p2gxUgmMSTjlHMvzu59B1q9JgAATANBi5FNImypbgEAMI8ELdZlEmEraapbwhYAAPNC0GLdNi0dlysvPKPViYSJVkIAAOaHoMXYJrEnllZCAADmgaDF2CbVRphoJQQAYLYJWhyVSbURJloJAQCYXaXW2vUapk4pZWuSPXv27MnWrVu7Xs7M2H/wWznl3Isncu1PPf/0ibQqAgDAavbu3Ztt27YlybZa695R36eiRWsm3UqougUAwKwQtGjVJMPWqedf4r4tAABmgqBF6yZd2TKVEACAaSdoMRH9IRlaCQEAWESCFhM16VZC1S0AAKaRoMXETTJsJfbcAgBg+ghabAhhCwCARWIfrSHsozVZu/cdyKnnXzKx69tzCwCAtthHi5mxfcsJE69uGZQBAECXBC06MelWQoMyAADokqBFZyYdthL7bgEA0A1Ba5odvCk5b1vzOHhT16uZiI0IW4lhGQAAbKzOg1Yp5WmllC+UUg6UUj5cSvnRVc49s5RShzxOGDjnvCGvf2VjPg3jELYAAJg3nQatUsovJnlJkhckuX+S9yZ5RynlLqu8bW+SOw4+aq3LJx/887JzfqDlpW+8C3bMbVUrEbYAAJgvXVe0npnk1bXWP6y1Xl5rfUaSq5M8dZX31FrrVwYfQ8751rJzrp/I6mnVpqXjcuWFZ+Syc06b6PcRtgAAmLTOglYpZSnJA5O8c9lL70zy0FXeuqWUclUp5ZpSyttLKfcfcs49SinX9loS/7SUcvc11nJ8KWVr/5Hkluv6MBtlzqtafRuxB5YhGQAATFKXFa3tSY5Nct2y49clucMK7/l0kjOT/EySxyY5kOR9pZR7DJzzwSS/kuT0JE/pXev9pZTbrrKWZyfZM/C4Zj0fhHb1K1sb1Upozy0AANrWdetgktRlz8uQY82JtX6g1vrHtdaP11rfm+TRST6b5DcGznlHrfUttdZP1FrfleSM3ktPWGUNL0yybeBx5/E+ygZYkKpW0gSuSbcRJvbcAgCgfV0Grd1Jvp0jq1e3y5FVrqFqrd9J8o9J7rHKOTcl+cQa59xca93bfyS5cZTvP3FLm5PnXNv1Kjq1EW2Eff12QhUuAACOVmdBq9Z6MMmHkzxy2UuPTPL+Ua5RSilJ7pfky6ucc3ySe612zsxZsKrWlReesWGthIkKFwAAR6/r1sEXJXlyKeWJpZR7lVJenOQuSV6ZJKWU15dSXtg/uZTy3FLK6aWUu5dS7pfk1WmC1isHztlVSnl4KeVupZQfSvLmJFuTvG4DP1d7VqpqHdy/8Wvp2EaNgO9z/xYAAOPauL6sIWqtb+oNqTg3zX5Xn0zyU7XWq3qn3CXJdwbecqskr0rTbrgnyUeTPKzW+qGBc+6c5E/SDNu4PskHkjx44JrMsH6Fa//Bb+WUcy+e+Pc79fxLkiSfev7pG9rGCADAbCu1Dp07sdB6I9737NmzJ1u3bu16OY2DNzUtg4Oec21T8VpQu/cd+G4Q2igCFwDAYtm7d2+2bduWJNt68xxG0nXrIKNa2pw88zNdr2KqbN9ywoa2EiY2OwYAYDSC1iw55tjDny/QUIyVbPR9W4nNjgEAWJugNUuWNnW9gqnUv29rI/bcGqS6BQDAStyjNcRU3qPV516tNW3UoIxB7t0CAJhP7tFaZAs46n01XbUTqm4BANAnaM2aYftqfevmbtYyxTYtHddJK6F7twAASASt+fCS71/4oRjDdDGVMBG4AAAQtGbT0ubkrCsOP2YC4VBdDcpItBMCACwyQWtWmUC4Ll1Xt1S4AAAWi6mDQ0z11MFBJhCOpYuphINMKAQAmB2mDtLQQrimLqYSDjrl3Iuze9+Bzr4/AACTp6I1xMxUtBJVrRZ0WeFS3QIAmG4qWhxiX6116WIUfJ8JhQAA80nQmnXD9tXatVP74Dp1NSyjz4RCAID5ImhBT5ej4BPVLQCAeSJozQP7arVq+5YTcuWFZ3RW4VLdAgCYfYLWvBi2r5awdVS6nE6ougUAMNsELVhFv52wy8BlFDwAwOwx3n2ImRrvPmjYqPezrki2nNjNeubQ7n0Hcur5l3TyvY2CBwDYeMa7M3wC4Vcv72Ytc6rL6YTaCQEAZoeK1hAzW9Hq23d9M+K9zwbGE9HlRseJChcAwEZQ0WJlhmJMRJfDMhL3bwEATDP/OXweDZtAyET0h2Uk3VS4+veLqW4BAEwXFa15NOxeLVWtiZuGcfAqXAAA08E9WkPM/D1ayfAJhIn7tTZQlxMKE1UuAIA2uEeLww2rarGhupxQmDRVLhMKAQC6oaI1xFxUtPqWV7Z+67PJLW/f3XoWlAmFAACzSUWL0fz+97pXqwMmFAIALBYVrSHmqqKVHLmvVuJerQ51Xd1KksvOOS3bt5zQ6RoAAGbBuBUtvUSLwLj3qdIfCd/lsIzB76utEACgfVoHF4Fx71Op62EZfdoKAQDap3VwiLlrHUyMe59yXY+C71PdAgA4nGEYrG6lce8H92/8WjjCNFW3Tj77ImPhAQCOkorWEHNZ0UpUtWbItFS4ElUuAGCxqWixNpsYz4xpqXAlqlwAAONQ0RpibitafTde1+ynNUhVa2pNwzj4QSpcAMAiGbeiJWgNMfdBa9i+WomwNQOmJXQJWwDAotA6yOhW2lfLYIypt2npuKloKey3E2opBAAYTtCCGdPf8PjKC8+YmtAlbAEAHE7r4BBz3zqYmEA4Z6alpfCyc07L9i0ndL0MAIDWuEerRQsRtBJhaw5Ny1h493ABAPPCPVqs39Lm5Kwrjjx+wY4mhDFzpmUsvJHwAMCi85+cF91KgzGYWf17uJLuK1z9dkYVLgBg0ahoLbqVNjFW1ZoL01Th2r3vQNfLAADYMO7RGmJh7tEaNOx+rbOuSLac2M16aN20DMxIVLgAgNlhGEaLBK0BBmPMHYELAGB0hmFwdLQQLoz+PVyXnXNa10uxBxcAMLdUtIZYyIpW377rk107Dz+mqjXXpqXCZQ8uAGAaaR1skaC188jjwtbc63pC4SAthQDAtBC0WrTQQWule7USYWtBTEuFKxG4AIDuCVotWuiglawctkwhXCjTFLi0FQIAXRG0WrTwQSsxhZDvmqbAlahyAQAby9RB2mUKIT2blo6bik2P+0459+KcfPZFphUCAFPNfxYG1tQfCZ9MT4WrvwZthQDANNI6OITWwQGmEDLEtISt5bQVAgBt0zrIZCxtGn784P6NXQdTpV/hmpaNj/tOOffi7N53oOtlAACoaA2jorWMwRisYRorXKpbAEAbTB1skaA1hLDFiKZp4+NE4AIAjo7WQbphCiHLbN9ygimFAMDCU9EaQkVrBaparJOWQgBg1mkdbJGgtYKVglYibLGmaQtdAhcAMAqtg0zeSpsYJ6YQsiYbHwMAi0RFawgVrTVoIaQF01bhsvExADCMihYbZ6XKlsEYrMO0VbhOPf8SFS4AoDUqWkOoaI1o3/XJrp1HHlfZYgzTNhZehQsASFS06MLSpuHH3a/FGKZtLLwKFwBwNFS0hlDRWgf3azEh01bhMqUQABaT8e4tErTWSdhiQqZtYEYicAHAotE6CMydTUvH5coLz5iqlsJTzr04u/cd6HoZAMCUU9EaQkVrDKpabIBprHAlqlwAMM9UtJhORr7Ton6F67JzTut6KYc55dyLDc0AAA6jojWEitaYVqpqnXVFsuXEjV8PC2Eaq1wqXAAwPwzDaJGgdRS0ENKBaQxbfUIXAMw2QatFgtZRErboyDQHrkToAoBZJGi1SNBqwb7rk107Dz8maLFBBC4AoC2CVosErRaoajEFpj1wXXbOadm+5YSulwEArELQapGg1RJhiykw7WFrkEoXAEwfQatFglaLhoUtUwjpiNAFAKyXoNUiQatlw8KWqhYdmaWw1Sd0AUB3BK0WCVoTYDgGU2j3vgM59fxLul7GughdALCxBK0WCVoT4H4tppgqFwCwknGDlv+XplsH9wtadG7T0nG58sIzksxO6OqvUeACgOmkojWEitaEDKtq/ex/T+7/uG7WA2uYpdZCgQsAJkPrYIsErQm78avJ79/j0PNf/8dk+/d2tx5Yw6xUufqELgBoj6DVIkFrwoZVts7+YnLCtm7WAyMSuABg8QhaLRK0NoCR78y4WQpdl51zWrZvOaHrZQDATBK0WiRobRBhiznhXi4AmF+CVosErQ20PGz92vuSO9y7u/XAUZilKlcidAHAKAStFglaG+zrVycvHQhXqlrMuFkLXInQBQArEbRaJGhtMC2EzKlZDFzu5wKAwwlaLRK0OiBsMedmMXSpcgGAoNUqQasjB/YmF5506PlvX5FsPrG79cAEzGLgSoQuABaXoNUiQatDN3ydc6s5AAAgAElEQVQhedn9Dj1X1WKOzVro0lYIwCIaN2gdM7klwRhuc7fk0W849Py9L+puLTBhm5aOy6eef3rXyxjZqedfkpPPvignn31Rdu870PVyAGCqqWgNoaLVseX3az3uLck9/m1364ENNEt7cvVpKwRgnmkdbJGgNSXe+rTkY29svv6dq5Jb3Krb9cAGm6XQpa0QgHklaLVI0JoS+29IXnFqsv9ryWnPTX70mV2vCDoxS/dyqW4BMG8ErRYJWlPkstckb39G8/WT35Xc+UHdrgc6NEuBKxG6AJgPglaLBK0pcvO+5IV3ar4+8V7JUy4xhRAyW22FidAFwOwStFokaE2Zr1+VvPQ+h57/zpXJLW7d2XJgmsxalSsRugCYLca7M79ufdfkCW879Px3T24mEwLZtHRcrrzwjFx2zmldL2Vkp5x7cfYf/FbXywCAiVLRGkJFa0r981uT//OEQ89tZgxHmMUKV6LKBcD00jrYIkFriv31s5IP/c/m6ye+M7nLD3W7Hphysxa8jIkHYNoIWi0StKbYgT3JhXdpvj7x+5KnvDtZ2tTtmmAGzFrg6hO8AOiaoNUiQWvK3fCF5GX3a76+z6OTn/9f3a4HZtAsBi+hC4AuCFotErRmwOfelbzx3zdf//wfJvf5D92uB2bYrI2KT9zTBcDGEbRaJGjNiHeek7z/5c3XT700uf0p3a4HZtwsVrkSoQuAyRK0WiRozYhvfr0Z9Z4kt//+5MmXJN9zi06XBPNgVgOX1kIAJkHQapGgNUN2fy55xanN16c+MXnUi7tdD8wZbYUALDpBq0WC1oy5/O3Jmx7XfP0zL08e8CvdrgfmlNAFwCIStFokaM2gi5+TXPoHzdeP//Nk52ndrgfm3Ky1F2orBGBcglaLBK0Z9J3vJH/ymORzvb/4PeldyUkP6nZNMOdmLWz1qXIBsB6CVosErRl10+7k9/7Noef/+VPJtjt1tx5YMLPWWihwATCKcYPWMZNbEmywzduT3/z4oecvPiW5eV9364EFs33LCfnU80/vehkjO+Xci3Py2Rdl974DXS8FgDmkojWEitaMu+r9yWt+8tDz51ybLG3ubj2wgGa1rTBR6QLgcFoHWyRozYH/+5LkXc899FzYgs4IXQDMMq2DMOiHn57c45GHnmshhM5sWjpuploKB/XbC/cf/FbXSwFgxqhoDaGiNSe+cXXyknsfev7sa5Ljb9ndeoAks13hMiYeYPFoHWyRoDVHvvDe5HWPOvRcCyFMlVkOXYnWQoBFoHUQhrnbjyY/84pDzy/YkRy8qbv1AIeZ5bbCpGkt1FYIwDAqWkOoaM2hv3p68pHXNl8/5T3Jne7f5WqAFcx6hStR5QKYN1oHWyRozaE9X2r21ep71r8km27b3XqAVc1D4HI/F8B8ELRaJGjNoYM3NW2Dg86+OjnB7xem2TwErkSVC2CWCVotErTm1LCwZTgGzJTd+w7k1PMv6XoZYxO4AGaPoNUiQWvOfex/J2996qHnwhbMlHmocmkrBJgdglaLBK0F8I7fST74yubrn391cp9f6HY9wFjmIXQlKl0A00zQapGgtQBuvjF54Z0PPf9P/5Dc8b7drQc4arMeuoQtgOkkaLVI0FoQ3/x68rsnH3r+zMuTrTtWPB2YHUIXAG0RtFokaC2QG65MXjZQyfqdK5Nb3Lqr1QAtm/XAlQhdAF0TtFokaC2Yqy5NXvMTh54/+5rk+Ft2tx5gIoQuAMYhaLVI0FpAn3hz8pYnNV8/8MzkUS9JSul0ScDkGBMPwKgErRYJWgvIHluw0GY1eBkTDzB5glaLBK0F9t4XJZc879BzYQsWyiy3F6pyAUyGoNUiQWvB/eWvJx99Q/P1T/1+8oNP7nY9QCdmOXQlghdAWwStFglaC+7AnuTCuxx+TGULFtoshy6BC+DoCFotErQ4YkPjRNgCkszu/VyJ0AUwDkGrRYIWSZKb9yUvvNPhx4QtoGeWq1yJQRoAoxK0WiRocZi3/efkw3/UfP3YNyX3/InVzwcWyqwHrkSlC2A1Mx20SilPS/LbSe6Y5J+TPKPW+t4Vzj0zyWuGvHSLWuuBca455HsIWhxyYG9y4UmHnj/1/cntv7+79QBTS+gCmD8zG7RKKb+Y5A1JnpbkfUn+U5InJzml1vrFIeefmeSlSe45eLzW+pVxrznkewhaHG7f9cmunYeen3VFsuXE7tYDTD2hC2A+zHLQ+mCSj9Ranzpw7PIkb621PnvI+WcmeUmt9VZtXXPI+wUtjnTDvyQvu/+h57/1ueSWt+tuPcDMELoAZte4QeuYyS1pbaWUpSQPTPLOZS+9M8lDV3nrllLKVaWUa0opby+lfPdvv+Ncs5RyfClla/+R5Jbr/SwsgNvcPXn0Gw49f/9Lu1sLMFM2LR2XTz3/9K6XcVROOffinHz2Rdm978DaJwOQrv/T1PYkxya5btnx65LcYYX3fDrJmUk+kWRrkqcneV8p5b611s+Nec1nJ3nuehfPAtp52qGvL31FcrtTkvs/rrv1ADNj09JxufLCM5LMdoVr+Wh70wsBhuu0dbCUsiPJl5I8tNZ66cDx/5Lkl2ut3zfCNY5J8pEk/1Br/c1xrllKOT7J8QOHbpnkGq2DDHXwpuSCHYee/+rfJzvu1916gJk3y3tzDdJeCMyjmWwdTLI7ybdzZKXpdjmyIjVUrfU7Sf4xyT3GvWat9eZa697+I8mNoy2fhbS0OTl7YKbKqx6e7Ptqd+sBZt72LSfMfGthcqi9sP/Yf/BbXS8JoDOdBq1a68EkH07yyGUvPTLJ+0e5RimlJLlfki+3dU1Y0wnbkt/8+KHnu+6RfPMb3a0HmHn91sIrLzxjLkJXcih4CVzAIpqGqYP9Uey/luTSJL+a5ClJvr/WelUp5fVJvtSfFlhKeW6SDyT5XJp7tH4zyS8n+eFa64dGueYIazJ1kNF8+h3Jnz7m0POzr05O8GcGaNe8tBb2aTEEZsm4rYOd/69crfVNpZTbJjk3zebCn0zyUwOB6C5JvjPwllsleVWa1sA9ST6a5GH9kDXiNaEdd3/Y4c8vPCl59peS47d0sx5gLm3fckKuvPCMmR6iMWj5ZxC8gHnUeUVrGqlosS7Lh2MkwhYwcfMSupYTuoBpM7MbFk8jQYt1Gxa2zroi2XJiN+sBFs68BS9j44FpIWi1SNBiLMPC1nOubaYUAmwgoQugPYJWiwQtxiZsAVNk3gLXIC2GwEYRtFokaHFUbt6XvPBOhx8TtoCOCV0A4xG0WiRocdQO7G0mEA4StoApMm8j4/uELqBtglaLBC1asfcryYvuefgxYQuYMipdAKsTtFokaNGKYfdrJcIWMJXmtcI1SPACxiFotUjQolXaCIEZNM/BS+AC1kPQapGgRev235D8t7sdev6r/5DsuG936wEY0Ty3FhobD4xC0GqRoMVE7P588ooHHHr+zMuTrUNaCwGm0DwHrj6VLmAYQatFghYT4Z4tYI7Me/ASuoA+QatFghYTI2wBc2beA1cidMGiE7RaJGgxUcPC1llXJFtO7GY9AC2a1yEa7ueCxSVotUjQYuKGhS1VLWDOzHu1S6ULFoOg1SJBiw2x7/pk184jjwtcwBya59Cl2gXzbdyg5T/DQFeWNg0/fnC/oAXMnU1Lx+XKC89IMn+ha3mrpOAFJCpaQ6losWEMxwAW2LwFrmG0F8Ls0zrYIkGLDSVsASSZ/+AldMFsErRaJGix4YQtgO8SuIBpImi1SNCiEyuFLaPfgQU174ErEbpgFghaLRK06IzKFsBQ87o/V5/ABdNL0GqRoEXnvvLPySsfevgxYQtAlQvYcIJWiwQtOmePLYA1LULoSgQv6Jqg1SJBi6mgjRBgJAIXMEmCVosELaaGsAWwbosQvIQu2DiCVosELaaKNkKAsc176BK4YPIErRYJWkyVlapaidHvAGOY5/AleEH7BK0WCVpMJW2EAK0TuoC1CFotErSYWsIWwMTNa/gSvGA8glaLBC2mmnu2ADbEvAauROiC9RC0WiRoMdVWu2dL2AKYmHkOXonwBSsRtFokaDH1VgpbhmMAbIh5Dl0CFxxO0GqRoMVMUNkCmBq79x3Iqedf0vUyJkLwYtEJWi0StJgZwhbA1Jnn0JUIXiweQatFghYzZaXhGNoIAabCPAcvoYtFIGi1SNBi5hj7DjAT5vnerkTwYj4JWi0StJhJwhbAzJnnalef8MWsE7RaJGgxs1ZqI0wELoApN+/VrkToYjYJWi0StJhpRr8DzIV5r3YJXcwKQatFghYzTxshwFxYhCpXn+DFtBK0WiRoMReELYC5InRBNwStFglazI2V7tkStgBm3ry3FvYJXXRN0GqRoMXcsKExwEIQumByBK0WCVrMFWELYKEsSouh0MVGEbRaJGgxl7QRAiwcoQuOnqDVIkGLubRaZcvod4CFsAjBS+iibYJWiwQt5ppphAD0LELwSpLLzjkt27ec0PUymFGCVosELeaesAXAEIZqwJEErRYJWiwE92wBsIJFqXQlql2sTdBqkaDFwlDZAmANixS6EtUujiRotUjQYqEIWwCMSOhiEQlaLRK0WDjCFgBjWpTwJXQtLkGrRYIWC8k9WwC0ZN7Dl9C1WAStFglaLCyVLQBaNu+hKxG85p2g1SJBi4UmbAEwIYsQuvqEr/khaLVI0GLhaSMEYMKELmaFoNUiQQuisgXAhhG6mGaCVosELegZFrae8NfJ3X64m/UAMPeELqaNoNUiQQuW+dq/JC+//+HHVLYA2AC79x3Iqedf0vUyNoTgNZ0ErRYJWrCMNkIApsSiVLyErukhaLVI0IIhhC0AppDgxaQJWi0StGAFwhYAU07wom2CVosELVjDVz6ZvHLZQAxhC4ApsiiBq0/wmhxBq0WCFqxBZQuAGSN4MS5Bq0WCFoxgWNj69Q8n24dsdAwAU2aRgpfQdXQErRYJWjCiYWHrd65KbnGrbtYDAGNalOAldK2foNUiQQvWQRshAHNmUUJXIniNQtBqkaAF6yRsATCnhC4ErRYJWjAGYQuABbMoIWzRA5ig1SJBC8YkbAGwYBYlbPUtYugStFokaMFR2Hd9smvI5EFhC4AFsSjha1FCl6DVIkELjsJKVa1E2AJgoQhc80HQapGgBS3QRggA37Uooeuyc07L9i0ndL2MVglaLRK0oCUrtREmAhcAC28Rwtc8VLvGDVqz/amB6ba0aeXXDu4XtABYaJuWjsuVF56RZH5D1/LPNA/Ba1QqWkOoaEGL3LMFAOsyr6FrmFkIXloHWyRowQQIXAAwlt37DuTU8y/pehkbYhqDl6DVIkELJsQ9WwDQikWpek1D8Bo3aB1ztN+4lLK1lPJzpZR7He21gDm31j1bAMBI+vd3fer5p3e9FFaw7opWKeXPkvxDrfUVpZRbJPl4kpOTlCSPqbW+pfVVbjAVLZgwo98BoHXzWOWa5YrWOKt+WJIX9L7+d2kC1q2SPCHJOUlmPmgBHemHL4ELANZtcIphMp/Ba5aME7S2Jbmh9/VPJHlLrXV/KeWiJL/X2sqA+bW0OTlvz8r3bBn9DgBHbXnw6hPANsY4QevqJA8ppdyQJmg9pnf81kkOtLUwYIHt2qmqBQATsgj7d02DcYZhvCTJG5Nck+TaJO/pHX9Ykk+0syxgIWw5sQlUw1ywo7mXCwCYmH7oMlijfWONdy+lnJrkpCR/W2vd1zt2RpJv1Frf1+4SN55hGLDBVttjK1HdAoANNi2Vrr//7Ufkrrft9u8AGzkMI7XWy5JcliSllGOT/ECS99davz7O9YAFt7Q5OeuKlffYcs8WAGyoaWkvvO3mpU6+bxvGGe/+kiSfqLW+uhey/j7JQ5PsT/KoWut7Wl/lBlPRgo6sVtlS1QKAqbFR4WuWx7uPE7SuSfJztdbLSik/l+QPkvxYkl9J8mO11h9e1wWnkKAFHRK2AGDmTCp4LVrQOpBkZ631mlLKq5Lsr7U+o5RytyQfr7XOfDIRtGAKrDT6XdgCgKnXVvCa5aA1zqqvS3JKKeXLaca7P613fFOSb49xPYAjLW0afvyCHc39XFtO3Nj1AAAjs3nyeOPdX5Pkz5J8MklN8re94z+U5NMtrQtYdEubVx79vmun0e8AMEMWcYz8uOPdfyHNePf/U2u9pnfsCWnGu/9lu0vceFoHYcq4bwsA5tJala5Fax1MrfXNQ469bpxrAaxptfHvRr8DwMya5xbDcStaD09yVpJ7pWkfvDzJ79Va39vu8rqhogVTSmULANhg41a01n2PVinl8UnelWbfrJcleUWSbya5pJTyS+u9HsDIVrtv64Id7tsCAKbGOOPdL0/yqlrri5cdf2aSp9Ra79Xi+jqhogVTTmULANggG1bRSnL3JG8bcvyvktxtjOsBrI/KFgAw5cYJWlcnOW3I8dN6rwFM3tLm5Lw9wwOXsAUAdGycoPX7SV5WSvkfpZRfLqU8vpTyyiQvTbKr3eUBrKE/kXA5YQsA6NC6x7vXWv9HKeUrSX4ryaN7hy9P8ovzsIcWMIOWNg0/fsEO92wBAJ0Ya7z70AuV8j1J7lhr/WIrF+yQYRgwgwzIAAAmYCOHYazklCRfaPF6AKPr37OljRAAmAJtBi2A7q3WRrjv+o1dCwCwsAQtYL6sNvp9106VLQBgQwhawPxZa/T7edsELgBgokaeOlhKuc8ap9zzKNcC0K7+6PddO498zURCAGCC1jPe/WNJapIy5LX+8XZGGAK0ZaV7tpLk4H5BCwCYiJHHu5dS7jrKebXWq45qRVPAeHeYQyuNf1fVAgBWMe5499b20ZonghbMKXttAQDrNA37aAFMt9UmEt5sOAYA0B5BC1gsK4Wt37+HaYQAQGsELWDxrFbZMv4dAGiBoAUsptXCVtJMJAQAGJOgBSyu1cLWrp0qWwDA2NY9dbCU8tEM3y+rJjmQ5Iokr621vvvol9cNUwdhAZlICAAMsZFTB/8myd2T3JTk3Unek2Rfkn+T5B+T3DHJu0opPzvGtQG6sbQ5OeuK4a9dsENlCwBYl3GC1vYkv19r/dFa62/VWp9Za31Ykl1JNtda/58k5yf5/9pcKMDEbTlx9SEZwhYAMKJxgtajk/zJkON/2nstvdfvOe6iADqz1kRCYQsAGME4QetAkocOOf7Q3mv969487qIAOrVW2Np3/cauBwCYOceN8Z6XJ3llKeWBae7Jqkl+MMmTk1zQO+f0JB9tZYUAXejfs7Vr55Gv9Y8ZkgEArGDdUweTpJTyuCS/nkPtgZ9J8vJa6//uvX6LJLXWemCFS0w1UweB71ptGmGfwAUAc2vcqYPjVLRSa31jkjeu8vo3x7kuwNTptxGuFrYO7he0AIDDjL1hcSnlgaWUx5dSHldKuX+biwKYKquNfk+aVkJDMgCAAeNsWHy7NBMGH5HkG0lKkm1p9tR6TK115u8S1zoIDKWNEAAWzkZuWPzyJFuTfH+t9Ta11lsnuXfv2MvGuB7AbFhtGmHfwf0bsxYAYKqNU9Hak+Tf1lr/cdnxH0zyzlrrrVpcXydUtIA17bt++ETCPpUtAJgLG1nROibJvw45/q9jXg9g9ixtWv11mxsDwEIbJxj9XZKXllK+e6NCKeVOSV6c5JK2FgYw1ZY2J+ftWb2VUNgCgIU1TtD69SS3THJlKeXzpZQrknyhd+w321wcwNRbayKhsAUAC2nd+2jVWq9O8oBSyiOTfF+aqYOfqrW+q+3FAcyEUdoI3bMFAAtl3cMwVrxQKScleV6t9YmtXLBDhmEAYzH+HQDmzkYOw1jJbZI8ocXrAcwW498BgB5TAgHatNY9W7t2Judtc98WAMw5QQugbVtOXLuydcGOZi8uAGAuCVoAkzDK+PddO1W2AGBOjTx1sJTy52uccqujXAvA/Om3Eu7aOfz1/vAMQzIAYK6sp6K1Z43HVUle3/YCAWbeqK2EqlsAMDdaG+8+T4x3ByZm3/UrV7cSlS0AmDLTMN4dgLWMsrmxyhYAzLypCFqllKeVUr5QSjlQSvlwKeVHR3zfY0optZTy1mXHX9s7Pvj4wGRWD7AOowzJuGCHEfAAMOM6D1qllF9M8pIkL0hy/yTvTfKOUspd1njfXZPs6p0/zN8kuePA46faWjPAUbO5MQDMtc6DVpJnJnl1rfUPa62X11qfkeTqJE9d6Q2llGOTvDHJc5P8ywqn3Vxr/crA44bWVw5wNNYKWzY3BoCZ1WnQKqUsJXlgkncue+mdSR66ylvPTXJ9rfXVq5zziFLKV0spny2l/K9Syu1WWcfxpZSt/UeSW476GQCOyqithDY3BoCZ0nVFa3uSY5Nct+z4dUnuMOwNpZQfTvKkJE9Z5brvSPK4JD+e5LeSPCjJ35VSjl/h/Gfn8FH114y4foB29APXWVcMf111CwBmStdBq2/5jPky5FhKKbdM8sdJnlJr3b3ixWp9U631olrrJ2utb0vyk0m+N8kZK7zlhUm2DTzuvP6PANACUwkBYC4c1/H3353k2zmyenW7HFnlSpJ/k+TkJG8rpfSPHZMkpZRvJblnrfXzy99Ua/1yKeWqJPcYtoha681Jbu4/H7g2wMbqV7YO3tSEqmEu2NFUvracuLFrAwBG1mlFq9Z6MMmHkzxy2UuPTPL+IW/5dJIfSHK/gcdfJXl37+urh32fUsptk5yU5MutLBxg0pY2r9xGmDSthCpbADC1pqF18EVJnlxKeWIp5V6llBcnuUuSVyZJKeX1pZQXJkmt9UCvHfC7jyTfSHJj7/nBUsqWUsquUspDSiknl1IekeRtaapnf9HJJwQYx5YT1x6SIWwBwFTqPGjVWt+U5BlpJgl+LMnDkvxUrfWq3il3SbMP1qi+nabq9ZdJPpvkdb1/PqTWemNb6wbYEGuNgDeREACmUqn1iJkTC6834n3Pnj17snXr1q6XA9DYd33TMjjMc65tQhkA0Kq9e/dm27ZtSbKt1rp31Pd1XtECYESrTSS8YIfx7wAwRQQtgFkx6ubGwhYAdE7QApg1a00kFLYAoHOCFsAsMpEQAKaaoAUwq0aZSOi+LQDohKAFMMvWCluJ6hYAdEDQAph1a92zlahuAcAGE7QA5sFa92z1Hdw/+bUAADYsHsaGxcBMO3hTU8FajQ2OAWAkNiwGoDHqfVtaCQFgYgQtgHk0SthKtBICwIQIWgDzapSwtWunqhYATICgBTDPtBECQCcELYB5N2ob4QU7kn3XT349ALAATB0cwtRBYG6NMpEwMZUQAHpMHQRgbaNsbpw0YUwrIQCMTdACWDSjbm4sbAHA2AQtgEW0tDk5b89ogzKELQBYN0ELYJGN0koobAHAuglaAItulFZCEwkBYF0ELQBsbgwALRO0AGjY3BgAWiNoAXDIqJsbH9w/+bUAwAyzYfEQNiwGyGibG9vYGIA5Z8NiANo1aiuhNkIAOIKgBcDKhC0AGIugBcDq+psbr7bflrAFAIcRtAAYzdKm1V83kRAAvkvQAmA0/cqWzY0BYE2mDg5h6iDACPZd32xivBpTCQGYcaYOArCxtpxoUAYArEDQAmB8phICwFBaB4fQOggwBq2EAMwhrYMAdEsrIQB8l6AFQHu0EgJAEkELgLYJWwAgaAEwAaOGLRscAzCnBC0AJmM9GxwLWwDMGUELgMla2pycdcXq5whbAMwZQQuAyRt1IuG+6zdmPQAwYYIWABtjlPu2du103xYAc0HQAmDj9O/b0koIwJwTtADYeFoJAZhzghYA3dBKCMAcE7QA6M4oYStR3QJg5ghaAHRr1P22du1U2QJgZghaAEyHUapbF+zQSgjATBC0AJge62klFLYAmGKCFgDTZdRWQmELgCkmaAEwnZY2j7bfllZCAKaQoAXA9Bplv63EVEIApo6gBcB0G/W+LVMJAZgipdba9RqmTilla5I9e/bsydatW7teDgB9+65vAtVannNtE9AA4Cjt3bs327ZtS5Jttda9o75PRQuA2bG0abTzDMoAoGOCFgCzY9SJhImwBUCnBC0AZs969tsylRCADghaAMymUcNWkhzcP9m1AMAyghYAs2s9EwlVtgDYQIIWALOtf9/WWpsbJ+7bAmDDCFoAzIf1TCS0uTEAE2YfrSHsowUwww7e1ISpUdhvC4A12EcLAJL1DcnQSgjAhAhaAMwfYQuAjmkdHELrIMAc0UoIwFHQOggAw6y3umVQBgAtELQAmH/rCVv23AKgBYIWAIuhv9+W6hYAG0DQAmCxLG0ebXPjpKluqWwBMAZBC4DFs+XE9VW2tBICsE6CFgCLaT33bSVaCQFYF0ELgMXVv29LKyEALRO0AEArIQAtE7QAIFl/dUsrIQCrELQAYNB6qltaCQFYgaAFAMutZ1CGVkIAhhC0AGCYcVoJhS0AegQtAFjN0qbRz1XdAqBH0AKA1fQrW+vdc0vYAlhoghYAjEIrIQDrIGgBwHpoJQRgBIIWAKyHVkIARiBoAcA41jMCPhG2ABaMoAUA41pvdUvYAlgYghYAHK31bnC87/rJrgeAzglaANCG9YStXTtVtgDmnKAFAG1Zb2XLREKAuVVqrV2vYeqUUrYm2bNnz55s3bq16+UAMIsO3tSEqVE959omqAEwVfbu3Ztt27YlybZa695R36eiBQCTYCohwEITtABgUsaZSmhQBsBcELQAYNKWNidnXTHaubt2uncLYA4IWgCwEbacaM8tgAUiaAHARrLnFsBCMHVwCFMHAZi4fdc3bYLrYTIhwIYzdRAAZsmWE9c3lTDRTggwQwQtAOjKekfAJ8IWwIwQtACgS8IWwFwStACga+vdbysxKANgyglaADAt1lvdsucWwNQStABgmmglBJgLghYATButhAAzzz5aQ9hHC4Cpst49t+y3BdAa+2gBwLxa755bWgkBOidoAcAsWO+9WxfsMCgDoEOCFgDMCoMyAGaGoAUAs8SgDICZIGgBwCzqB66zrgdJQFUAABwoSURBVBjt/F07VbYANpCgBQCzbD2DMty3BbBhBC0AmHXrrW5pJQSYOEELAObFeqpbu3aqbgFMkKAFAPNknDHwqlsArRO0AGDeGJQB0DlBCwDmlUEZAJ0RtABgnq133y2thACtELQAYBEsbV5fK+H/397dR1t21vUB//5MnCgzmUFlMAQMWCMRBYUVUUFBJAYrdrkAW8mqttIXtLjQqkQxKHEsGEAiRkHrSy0pIIqtBYuIvKiAArpMIkIEGqMkNeSFQWQmE0gC4ekfe184OXPmzj3n7nvP2+ez1l733v1yzj7Jk8z87vd5flu6BbAtCi0AWBfTTCVMTCcE2AaFFgCsk2kbZSRdwaXYApiKQgsA1pF0C2BHKbQAYF1N+8ytJLnzYztzLwArplpr876HhVNV+5McOXLkSPbv3z/v2wGAnXfscNcEY6uedWNXqAGsuKNHj+bAgQNJcqC1dnSr10m0AABTCQEGptACADrTPnMr0SgD4AQUWgDA3U27dkuxBXAchRYAcLxp061LzuzWeQGQRKEFAGxmz96tP3Pr0rOt2wLoKbQAgM1plAEwNe3dJ9DeHQA2MU0reG3ggSWnvTsAsDv23GPr50q3gDWl0AIApjNrG3jNMoA1otACAGYzbRt4zTKANaLQAgBmJ90CmEihBQBsn3QL4G4UWgDAMKYtthLNMoCVpdACAIYzy1TCpCu4FFvAClFoAQDDmzXdUmwBK0KhBQDsDFMJgTWm0AIAdo6phMCaqtbavO9h4VTV/iRHjhw5kv3798/7dgBgddx5W1dETeNZN3YFG8AcHD16NAcOHEiSA621o1u9TqIFAOwea7eANbEQhVZVfX9VfaCqbq+qK6vqUVu87oKqalX1mrH9VVWHqurGqvp4Vb2lqr5iZ+4eAJjKxnTCC6/d+jXWbgFLZu6FVlU9OcllSX4mycOS/GmS11fVWSe57v5JLu3PH/djSX4kydOTPDzJzUneVFWnD3jrAMB27Dso3QJW1tzXaFXVXyS5qrX2tJF970vymtbaRSe45pQkb03y0iSPSnLP1toT+mOV5MYkl7XWXtDvOy3JLUme2Vr71S3ckzVaALCbpl27Zd0WsEuWco1WVe1Jcm6SN44demOSR25y6cVJDrfWfmPCsS9Ocsboa7bW7khXmE18zao6rar2b2xJJF8AsJv27DWVEFgp8546eK8kp6RLm0bdkq5YOk5VfX2S/5DkqSd4zY3rtvyaSS5KcmRku2HTuwYAhmcqIbBC5l1obRifv1gT9qVfY/WKJE9trX14iNfsPS/JgZHtfie7YQBgB2iUAayIU+f8/h9OcleOT5runeMTqST5kiQPSPLabilWkr5YrKpPJjknXeOL9K950xZec2Nq4R0bP4+8NgAwD3vuMf01l5zZFWj7Dg5/PwBTmmuh1Vq7s6quTHJ+klePHDo/ye9NuOT9SR4ytu+56dZU/eck/5DkE+mKrfOT/FXy6bVg35jkmUPePwCwQzaSrWS6RhmXnt191SwDmLNFmDr4oiT/sar+fVU9qKp+PslZSX4lSarqZVX1vCRprd3eWrt6dEvy0SS39j/f2bo2ipcleVZVPbGqHpzk8iQfS/LKOXw+AGA7pm2UkVi7Bczd3Aut1tqrkvxQuk6C70ry6CSPb61d359yVpL7TPmyP5uu2PrlJFckuW+Sx7XWbh3kpgGA3TVro4xjh3fmfgBOYu7P0VpEnqMFAAvs2OHPTBHcKlMJgRkt5XO0AACmNmujDOkWsIskWhNItABgCUzTJGOUdAuYwqyJlkJrAoUWACyRWaYSJgouYEtMHQQA1tMsjTISnQmBHSXRmkCiBQBLynRCYGASLQCAWZ65lWiWAQxOoQUArJZZpxJeenZy6IDphMAgTB2cwNRBAFgRs04lTEwnBJKYOggAcLw9e2dLtxLNMoBtkWhNINECgBUk3QJmINECANiMdAvYRRKtCSRaALAGtIIHtkCiBQAwjVkTLukWsAUSrQkkWgCwZqRbwAlItAAAZrWddMuDjoEJFFoAAMnsxZYHHQMTmDo4gamDALDmtIIHeqYOAgAMRSt4YJsUWgAAk+zZmxw6ojMhMBOFFgDAZjTKAGZgjdYE1mgBABNpAw9rZ9Y1WgqtCRRaAMCmjh3uug1OS8EFS0czDACA3bLvoLVbwKYkWhNItACALTOdEFaaRAsAYB620yzDg45hZUm0JpBoAQAz8aBjWDkSLQCAefOgY6Cn0AIAGJIHHQMxdXAiUwcBgMFoBQ9LzdRBAIBFpBU8rCWJ1gQSLQBgR2gFD0tHogUAsOi20wpeugVLRaI1gUQLANhx0i1YChItAIBlsp1069jh4e8HGJREawKJFgCwq6RbsLBmTbQUWhMotACAudAKHhaOqYMAAMtOK3hYGRKtCSRaAMDcSbdgIUi0AABWyb6DyaEj0ydc0i1YCAotAIBFNkt3wkvOTA4dUHDBHCm0AAAWnVbwsHSs0ZrAGi0AYGFpBQ+7yhotAIB1sGdvt3brwmunu87aLdhVCi0AgGU0Syt4a7dg15g6OIGpgwDAUtEKHnaMqYMAAOvKg45h4Ui0JpBoAQBLa9ZmGYmECyaQaAEAMHsr+ETCBQOSaE0g0QIAVoJ0C7ZNogUAwN3N2go+kW7BNim0AABW3XaaZWgHDzNRaAEArIPtplvHDg9/T7DCFFoAAOtk1nTr0rOlWzAFzTAm0AwDAFgbHnYMm9IMAwCA6XnYMewIidYEEi0AYC3Nmm5deG1XsMEKkmgBALA91m/BYCRaE0i0AIC1t52HHUu4WCGzJloKrQkUWgAAve0UXBpmsAIUWgNSaAEAjFFwsaas0QIAYOfs2Tvb+q1Eh0LWkkRrAokWAMAmpFusEYkWAAC7Q7oFJyXRmkCiBQCwRdItVpxECwCA3bdnb3LoSNfSfVqXnNk9JBlWkEILAIDt87BjuBtTBycwdRAAYJuOHe6KqGl40DELyNRBAAAWxywJl3SLFSLRmkCiBQAwoFnSrUSzDBaCRAsAgMU06/otreBZYhKtCSRaAAA7ZNZ28NIt5kSiBQDA4pv1YcdawbNkJFoTSLQAAHaBdIslINECAGC5SLdYYRKtCSRaAAC7TLrFgpo10VJoTaDQAgCYE63gWTCmDgIAsPz2HUwOHZl+SqFW8CwYidYEEi0AgAUxS8Il3WJAEi0AAFbPLA871iyDBSDRmkCiBQCwgKRbzIFECwCA1TZrunXogPVb7DqJ1gQSLQCABacdPLtEogUAwPrYzsOOJVzsAonWBBItAIAlMmu6lUi4OCmJFgAA62nP3u7ZWxdeO/21nr/FDpFoTSDRAgBYYtZvMSCJFgAAJNtbv+X5WwxEojWBRAsAYEVIt9imWRMthdYECi0AgBUzy8OOEwUXpg4CAMAJ7TvYNcyY5YHHmmUwA4UWAADrY5b1W569xQxMHZzA1EEAgDVg/RZbYI3WgBRaAABrZNaC68JruymJrDSF1oAUWgAAa0jCxQQKrQEptAAA1pgOhYzQdRAAAIaw7+DsDzzWMIOeRGsCiRYAAElmT7es31oZEi0AABjarOnWpWdrCb/mJFoTSLQAADjOrM0yEuu3lphECwAAdtIsDzveYP3W2pFoTSDRAgBgU9KttaG9+4AUWgAAbMmszTISBdeSUGgNSKEFAMBUJFwryxotAACYF+u3GCPRmkCiBQDAzKRbK0WiBQAAi2DP3uTQkdkSrkvO7NZ9sfQkWhNItAAAGMysCZd0ayFItAAAYBHNun5LurXUJFoTSLQAANgR0q2lo737gBRaAADsqFmfv3Xhtcm+g8PfDydk6iAAACyLfQdnm0546dlawS8JidYEEi0AAHbNrNMJpVu7wtTBASm0AADYddZvLSRTBwEAYJltpzvhoQOmFC4YhRYAACyK7T7sWLG1MEwdnMDUQQAAFsKs0wkTUwoHYuogAACsmlmnEyYSrjmTaE0g0QIAYCF5/tauk2gBAMCq287ztzTM2FUSrQkkWgAALDzrt3aFRAsAANbJdtdvSbh2lERrAokWAABLR8K1IyRaAACwznQoXCgSrQkkWgAALDXp1mAkWgAAQMf6rbmTaE0g0QIAYKVIuGYm0QIAACazfmvXSbQmkGgBALCypFtTmTXRUmhNoNACAGDlHTucXHr2bNeuUcGl0BqQQgsAgLUh4dqUNVoAAMD0rN/aERKtCSRaAACsJenWcSRaAADA9uzZmxw6klx47fTXXnJmt+6LJBKtiSRaAACQ2ROuFUq3JFoAAMCw9uyVbs1IojWBRAsAAMas6fotiRYAALBzNtZvzdKhcA0TLonWBBItAAA4iTVZv+WBxQNSaAEAwBYdO5xcevb01114bbLv4PD3M7ClnjpYVd9fVR+oqtur6sqqetQm5z6pqq6oqo9W1W1V9a6q+jdj51xeVW1s+/Od/yQAALBm9h2cbTrhpWcnhw6s7AOP515oVdWTk1yW5GeSPCzJnyZ5fVWddYJLPtKf+4gkX5nkpUleWlXfMnbeHya5z8j2+OHvHgAA2Pbzt1aw4Jr71MGq+oskV7XWnjay731JXtNau2iLr3FVkte11p7d/3x5knu21p6wxetPS3LayK7Tk9xg6iAAAExpO90Jk4Vbw7WUUwerak+Sc5O8cezQG5M8cgvXV1Wdl+ScJG8bO/yYqvpQVV1TVb9eVffe5KUuSnJkZLthq58BAAAYsZ3uhMnKJFzznjp4rySnJLllbP8tSc440UVVdaCqjiW5M8nrkvxAa+1NI6e8Psl3JXlskmckeXiSP+6Tq0mel+TAyHa/6T8KAADwaUMUXEtcbJ067xvojc9frAn7Rt2a5KFJ9iU5L8mLqurvW2tvSZLW2qtGzr26qq5Icn2Sb0vyv49789buSHLHp9+8aoaPAAAAHGej4NrulMIlM+9C68NJ7srx6dW9c3zK9WmttU8l2Vhp966qelC66X9vOcH5N1XV9Um+dLs3DAAAzGCWgmuJH0U116mDrbU7k1yZ5PyxQ+cneccUL1W5ezOLux+s+oIkX5TkpmnvEQAAGNCevVufTvjR/7ez97KD5p1oJcmLkry8n973ziTfm+SsJL+SJFX1siQf3OhAWFUXJbkiyd8l2ZOubfu/TfK0/vi+JIeS/G66wuoBSS5Jl569epc+EwAAcCJbTbc+7/67d08Dm3uh1Vp7VZ84XZzueVdXJ3l8a+36/pSzknxq5JK9SX45XcOKjyd5f5LvHlmXdVeSh6Qrvu6Zrtj6kyRPbq3dusMfBwAA2KoVXr819+doLaKq2p/kiOdoAQDALhovuBbgmVqzPkdr7okWAABAks8kXCtg3s/RAgAAWDkKLQAAgIEptAAAAAam0AIAABiYQgsAAGBgCi0AAICBKbQAAAAGptACAAAYmEILAABgYAotAACAgSm0AAAABqbQAgAAGJhCCwAAYGAKLQAAgIEptAAAAAam0AIAABiYQgsAAGBgCi0AAICBKbQAAAAGptACAAAYmEILAABgYAotAACAgSm0AAAABqbQAgAAGJhCCwAAYGAKLQAAgIEptAAAAAam0AIAABiYQgsAAGBgCi0AAICBKbQAAAAGptACAAAYmEILAABgYKfO+wYW2dGjR+d9CwAAwBzNWhNUa23gW1l+VXXfJDfM+z4AAICFcb/W2ge3erJCa4KqqiRnJrl13vfCtpyermC+X/y7ZOcZb+w2Y47dZsyx2xZpzJ2e5MY2RfFk6uAE/T/ALVerLKauXk6S3NpaMw+UHWW8sduMOXabMcduW7AxN/X7a4YBAAAwMIUWAADAwBRarLI7kvx0/xV2mvHGbjPm2G3GHLttqcecZhgAAAADk2gBAAAMTKEFAAAwMIUWAADAwBRaAAAAA1NosVSq6qKq+suqurWqPlRVr6mqc8bOOa2qXlxVH66q26rq/1TV/cbOOauqXtsf/3BV/WJV7dndT8Oy6cdfq6rLRvYZbwyqqu5bVa+oqn+sqo9V1buq6tyR41VVh6rqxqr6eFW9paq+Yuw1Pq+qXl5VR/rt5VV1z93/NCy6qjq1qp5bVR/ox9PfV9XFVfVZI+cYc8ysqh7d/xl4Y/9n6BPGjg8yvqrqIVX11v41PtiP48ocKbRYNt+Y5JeSfF2S85OcmuSNVbV35JzLkjwxyQVJviHJviS/X1WnJEn/9XVJ9vbHL0jyHUl+bpc+A0uoqh6e5HuTvHvskPHGYKrq85K8Pcknknxrki9P8owkHx057ceS/EiSpyd5eJKbk7ypqk4fOeeVSR6a5J/320OTvHyn75+l9Mwk/yndeHpQuvH1o0l+YOQcY47t2Jvkr9ONn0m2Pb6qan+SNyW5sX+NH0hyYf+689Nas9mWdktyMElL8uj+5wNJ7kzy5JFzzkxyV5Jv6X/+1v7nM0fOuSDJ7Un2z/sz2RZvS1c8XZPkm5O8Jcll/X7jzTboluT5Sf50k+OV5KYkzxzZd1q6Quz7+p8f1P9/8WtHzvm6ft858/6MtsXakvx+kt8Y2/e7SV7ef2/M2Qbb+jHxhJGfBxlfSZ7WX3PayDk/nuSD6R9nNY9NosWyO9B//Uj/9dwkn53kjRsntNZuTHJ1kkf2ux6R5Op+/4Y3pPsP+9zA8X4pyetaa28e22+8MbRvT3JFVf3Pfnr0X1XVU0eOf3GSM3L3MXdHkrfm7mPuSGvtL0bO+fMkR0bOgQ1/luS8qnpgklTVV6VL3/+gP27MsZOGGl+PSPLW/toNb0j3y88H7NTNn8yp83pj2K5+3u2LkvxZa+3qfvcZSe5srf3T2Om39Mc2zrll9GBr7Z+q6s6RcyBJUlUXpCuIvnrCYeONof2zdL+ZfVGSS5J8TZJfrKo7Wmsvy2fGzC1j192S5P7992ck+dCE1/5QjDmO94J0v7R8f1XdleSUJD/RWvut/rgxx04aanydkeS6Ca+xcewD27rLGSm0WGYvSfKV6X7zdjKVLmLe0LZwDmuuqr4oyS8keVxr7fZpLo3xxmw+K8kVrbVn9T//Vb8o/GlJXjZy3vjYMeaY1ZOTfHeSf53kb9Ktfbmsqm5srf2PkfOMOXbSEONr0muc6NpdYeogS6mqXpxuis03tdZuGDl0c5I9/YLyUffOZ36zcXPGfsPWn//ZOf43Kqy3c9ONnSur6pNV9cl0DVl+sP/+lhhvDOumJO8d2/e+JGf139/cfx1PCcbH3BdOeO2DMeY43guTPL+19tuttfe01l6e5OeTXNQfN+bYSUONr+P+rO1fI5njGFRosVT6FqAvSfKkJI9trY1HwVem69Z1/sg190ny4CTv6He9M8mD+/0bHpfkjv562PBHSR6S7je8G9sVSX5z5HvjjSG9Pck5Y/semOT6/vsPpPsLxeiY25PuFwCjY+5AVX3NyDlfm2562MY5sOEeST41tu+ufObviMYcO2mo8fXOJI8ee3TK49J1Ibxup27+pObdfcRmm2ZL8svpusp8Y7rfXGxsnztyzn9N8g9JzkvysHR/WX5XklP646ckeU+SN/fHz+vPf/G8P59t8beMdB3sfzbebINt6doSfyLJs5KcnW46121JvmvknGf2/x98Yrqi/pXp/jJx+sg5r0/XTvnr+u3dSV47789nW7wtyeVJbkjybemaBjwxyeEkLxg5x5izzbyl69y78cvKluSH++/P6o9ve3ylK7pu7q99cP9aR5I8Y66ffd7/8G22abb+P9BJ21NGzvmcJC9O8o9JPpbktUm+aOx1zkrX0vZj/XkvzkhLUJvtRNuEQst4sw26JfkXfXF+e7ppg08dO15JDqWbZnh7uu5cDx475/OTvCLJ0X57RZJ7zvuz2RZvS3J6uucBXp/k40n+Lslzk+wZOceYs828JXnMCf7udnl/fJDxlW4Gytv617gpyU9ljq3dW2vdmwMAADAca7QAAAAGptACAAAYmEILAABgYAotAACAgSm0AAAABqbQAgAAGJhCCwAAYGAKLQAAgIEptABYa1V1XVX90LzvA4DVotACYC1U1VOq6qMTDj08ya/twvsr6ADWyKnzvgEAmKfW2uF538M0qmpPa+3Oed8HAJuTaAGwq6rqLVX1i1X1s1X1kaq6uaoObfHaA1X1a1X1oao6WlV/XFVfNXL8q6rqT6rq1v74lVX11VX1mCQvTXKgqlq/HeqvuVvS1B/7vqr6/ar6WFW9r6oeUVVn9/d+W1W9s6q+ZOSaL6mq36uqW6rqWFX9ZVV98+hnTnL/JD+/8f4jx76jqv6mqu7o7+UZY5/5uqr6yaq6vKqOJPn1qtpTVS+pqpuq6vb+nIum+hcBwI5SaAEwD9+T5LYkX5vkx5JcXFXnb3ZBVVWS1yU5I8njk5yb5Kokf1RVn9+f9ptJbkg3HfDcJM9P8okk70jyQ0mOJrlPv126yds9O8nLkjw0yfuTvDLJryZ5XpKv7s95ycj5+5L8QZJvTvKwJG9I8tqqOqs//qT+vi4eef9U1blJfifJbyd5SJJDSZ5TVU8Zu58fTXJ1/5mek+QHk3x7ku9Mck6S705y3SafB4BdZuogAPPw7tbaT/ff/21VPT3JeUnetMk135SuGLl3a+2Oft+FVfWEJP8y3Tqrs5K8sLX2/o3X3ri4T4Naa+3mLdzfS1trv9Nf94Ik70zynNbaG/p9v5AuIUu6F/3rJH89cv1PVtUT0xVDL2mtfaSq7kpy69j7/0iSP2qtPaf/+Zqq+vJ0hdXlI+f9cWvt04VhX8D9bZI/a621JNdv4TMBsIskWgDMw7vHfr4pyb1Pcs256ZKjf+yn5x2rqmNJvjjJxjS+FyX5b1X15qr68dHpfdu4v1v6r+8Z2/c5VbU/Sapqbz8V8r1V9dH+vr4sXeG3mQclefvYvrcn+dKqOmVk3xVj51yeLm37v/00zMed9BMBsKsUWgDMwyfGfm45+Z9Jn5WuIHvo2HZOkhcmSWvtUJKvSDfF8LFJ3tsnS9u5v7bJvo17fmGS70jyE0ke1d/Xe5LsOcn71Mhrje4bd9voD621q9IVmM9O8rlJfqeq/tdJ3guAXWTqIADL4qp067M+2Vq77kQntdauSXJNusYTv5Xk3yV5dZI7k5xyouu26VFJLm+tvTpJqmpfkgeMnTPp/d+b5BvG9j0yyTWttbs2e8PW2tEkr0ryqr7I+sOq+vzW2kdm+wgADEmiBcCyeHO6tVKvqapvqaoHVNUjq+q5fWfBz+078T2mqu5fVV+frinG+/rrr0uyr6rOq6p7VdU9Bry3a5M8qaoe2ndBfGWO/zP2uiSPrqr7VtW9+n0/l+S8qnp2VT2wqr4nydOzeaOOVNUPV9UFVfVlVfXAJP8qyc1JJj0nDIA5UGgBsBT6pg+PT/K2JP89XWr12+mSo1uS3JXkC9J1C7wmXTe/1yf5qf76dyT5lXQp0OF03Q6H8sNJ/ildd8PXpus6eNXYORf39/p3/ftvTAH8ziQXpOsq+F+SXNxau/wk73csyTPTrd36y/51H99a+9S2PwkAg6juzy0AAACGItECAAAYmEILgIVQVd812rZ9bPubed8fAEzD1EEAFkJVnZ7kC09w+BOtNQ/lBWBpKLQAAAAGZuogAADAwBRaAAAAA1NoAQAADEyhBQAAMDCFFgAAwMAUWgAAAANTaAEAAAzs/wP8ixMDOUwuzQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2418084cf98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "\n",
    "cvresult = cvresult.iloc[100:]\n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(100,cvresult.shape[0]+100)\n",
    "        \n",
    "fig = pyplot.figure(figsize=(10, 10), dpi=100)\n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators_detail.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "由图像可以看出，图像在我们设置的1000处停止。图二可以看出训练集和测试集上的标准差。\n",
    "因为程序运行较慢，就暂不增大n_estimators来找寻最佳参数，我们之后就按照n_estimators=1000进行进一步的参数调优。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第二步：调整树的参数：max_depth & min_child_weight"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    " (粗调，参数的步长为2；下一步是在粗调最佳参数周围，将步长降为1，进行精细调整)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "第一轮参数调整并未得到n_estimators的最优值，因此默认其参数为1000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': range(3, 10, 2), 'min_child_weight': range(1, 6, 2)}"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "max_depth = range(3,10,2)\n",
    "min_child_weight = range(1,6,2)\n",
    "param_test2_1 = dict(max_depth=max_depth, min_child_weight=min_child_weight)\n",
    "param_test2_1"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处考察模型数为3*4=12个模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.54223, std: 0.00228, params: {'max_depth': 3, 'min_child_weight': 1},\n",
       "  mean: -0.54356, std: 0.00221, params: {'max_depth': 3, 'min_child_weight': 3},\n",
       "  mean: -0.54467, std: 0.00253, params: {'max_depth': 3, 'min_child_weight': 5},\n",
       "  mean: -0.46003, std: 0.00190, params: {'max_depth': 5, 'min_child_weight': 1},\n",
       "  mean: -0.46961, std: 0.00307, params: {'max_depth': 5, 'min_child_weight': 3},\n",
       "  mean: -0.47640, std: 0.00258, params: {'max_depth': 5, 'min_child_weight': 5},\n",
       "  mean: -0.32893, std: 0.00119, params: {'max_depth': 7, 'min_child_weight': 1},\n",
       "  mean: -0.35636, std: 0.00204, params: {'max_depth': 7, 'min_child_weight': 3},\n",
       "  mean: -0.37544, std: 0.00281, params: {'max_depth': 7, 'min_child_weight': 5},\n",
       "  mean: -0.22746, std: 0.00464, params: {'max_depth': 9, 'min_child_weight': 1},\n",
       "  mean: -0.25699, std: 0.00400, params: {'max_depth': 9, 'min_child_weight': 3},\n",
       "  mean: -0.28209, std: 0.00416, params: {'max_depth': 9, 'min_child_weight': 5}],\n",
       " {'max_depth': 9, 'min_child_weight': 1},\n",
       " -0.2274649746202064)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #第一轮参数调整得到的n_estimators最优值\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch2_1 = GridSearchCV(xgb2_1, param_grid = param_test2_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch2_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch2_1.grid_scores_, gsearch2_1.best_params_,     gsearch2_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.227465 using {'max_depth': 9, 'min_child_weight': 1}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAELCAYAAAAoUKpTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucXFWZ7//PU12VvufenaQTMAGCJJAYoE0QBIISCMqEE2E0YOYHzBFUwo9BDyPgoPxAR8GZH8ZLVHDkIoIgIJyICAEFOTJC6GBACCKBYbDppLtzhdw6XVXP+aN2V+rWXdWX6uv3nVe9qvZl7Xpqp2s/e6+1ay1zd0RERLoSGugARERk8FOyEBGRvJQsREQkLyULERHJS8lCRETyUrIQEZG8lCxERCQvJQsREclLyUJERPIKD3QAfWXixIk+ffr0gQ5DRGRIWbdu3RZ3r8m33rBJFtOnT6ehoWGgwxARGVLM7L8LWU/VUCIikldRk4WZLTaz18xso5ldlWP5BWbWambrg8dnUpbFUuavLmacIiLStaJVQ5lZCbAKWAQ0As+b2Wp335Cx6r3ufmmOTex193nFik9ERApXzCuL+cBGd3/T3fcD9wBnFfH9RESkSIqZLKYCf0uZbgzmZTrbzF4ys/vN7KCU+WVm1mBmz5rZ/yhinCIikkcxk4XlmJc50tKvgOnuPhd4ArgjZdnB7l4PnAesNLNDs97A7OIgoTS0trb2VdwiIpKhmMmiEUi9UpgGNKWu4O5b3b0tmPwxcGzKsqbg+U3gKeDozDdw91vcvd7d62tq8t4mLCIiPVTM31k8D8w0sxnAO8AyElcJSWY2xd03BZNLgFeD+eOAPe7eZmYTgROAbxUxVhHpY+6Oe6I6IZ58nXhOLA/md6wbzKOT+R1l014D8bgnt5c23zve68C85HsWGgtOvOM9C4jFM7adOj/eWSzB8tTPEI+TFgsd83O8J+5MqCrlY3OmFO3/EoqYLNw9amaXAo8BJcCt7v6KmV0PNLj7auAyM1sCRIFtwAVB8VnAzWYWJ3H1c0OOu6gGra6+JJl/IPEcX454jj/GuKdsN3Neynumlo179h90PGP9ji9bR6xkxN2xPqnbS/1cKV+g7G0f+KM/sO0Df/AHtn3gs8Q7+dKl75P0efEggPS4E8tS487aBumfKXX/xr17++TAZ8+z7Rx/B6TOzyhHynYzt0eu/Z7cXu6/PXLNz4g78/+U1M9AYbFL/5p30NiiJwvzYfI/W19f7z35Bfe23fs54ztPZ32p4zm+MJnzsg66+qIMqJCBmSWeMTCSrzuWGWAdrw1CyXmJaSOYFywjWD+UsT4przuWkVG2Y3skYzoQQyhYmIyH9HKQHm8o5TVkx9qxDTLnp2yHjO10vGdyf+SIhcx1gVDowPYsVyxBoVyxW+Y+zdg+uWLPep3+nlj65yzOfkz9bLn/btL3Yyf7OeV13lhI/QxdxxIJhRhTEenmNya5f9YF7cNdGjbdffTUqHCIhYfXZhxAsr8wqX/AHV+WXAeY1PmZX5LMeYVtu+NAl/qH1sU8DnwGMg6SoVD6+rm+JJkHuoLi7uTL092DbtpnCJE9LyXOtMTQ8Y0SkaIZ8cmiqjTMjefMHegwREQGNfUNJSIieSlZiIhIXkoWIiKSl5KFiIjkpWQhIiJ5KVmIiEheShYiIpKXkoWIiOSlZCEiInkpWYiISF5KFiIikpeShYiI5KVkISIieSlZiIhIXkoWIiKSl5KFiIjkpWQhIiJ5KVmIiEheShYiIpKXkoWIiORV1GRhZovN7DUz22hmV+VYfoGZtZrZ+uDxmZRl55vZ68Hj/GLGKSIiXQsXa8NmVgKsAhYBjcDzZrba3TdkrHqvu1+aUXY8cC1QDziwLii7vVjxiohI54p5ZTEf2Ojub7r7fuAe4KwCy54OPO7u24IE8TiwuEhxiohIHsVMFlOBv6VMNwbzMp1tZi+Z2f1mdlA3y4qISD8oZrKwHPM8Y/pXwHR3nws8AdzRjbKY2cVm1mBmDa2trb0KVkREOlfMZNEIHJQyPQ1oSl3B3be6e1sw+WPg2ELLBuVvcfd6d6+vqanps8BFRCRdMZPF88BMM5thZqOAZcDq1BXMbErK5BLg1eD1Y8BpZjbOzMYBpwXzRERkABTtbih3j5rZpSQO8iXAre7+ipldDzS4+2rgMjNbAkSBbcAFQdltZvY1EgkH4Hp331asWEVEpGvmntUUMCTV19d7Q0PDQIchIjKkmNk6d6/Pt55+wS0iInkpWYiISF5KFiIikpeShYiI5KVkISIieSlZiIhIXkoWIiKSl5KFiIjkpWQhIiJ5Fa27DxGREcMdYu0Qa4PofojuO/A67bkNYsHyrHkpz9G2zpcly6fMm3QkLLurqB9RyUJEhp7Mg3POA+u+Tg62qQf0TpZ1bK87y/pKKAwlpRAOHiWjgudSCI9KPI+qhIoJB5ZNmNl3798JJQsRyc8d4tGMs99CD7adnGF3tSzr7DvHOtlD3PRM8uA86sBzuCzj4FwBJeNS1ilLXz/tuTT74J520A+2nzUvKBManK0DShYig1UsmnFAznXQzHVA7UV1RlfVIcU8OKcdUMsgUg5lY3MfbHtzIE4mgpRloZK++VzDnJKFSD7uiYNq+15o39PJc75le7tf1eHxvonfSvIfZMNlnR+c0w62XS3LPBB3khB0cB6SlCxkaItFe3Dgzrcsx/xun1UbRCoSZ8iR8uyz3XAplI3OcyDOVTXS1YE4V9WIDs7SN5QspDj65Gw8eL1/d471g9fx9u7HFook6qCTB/OUg3rZ5Ox5WevlW1aROEhbrqHkRYYmJYuRqM/Oxrtarydn42QfdDtel42B6sk9OHBnLiuHkkif71KR4U7JYjDp7dn4/gIO4L05G8918B1VAWVT+uBsPKiq0dm4yKCkZFGoXp2N56pG6WQbPdHZAblstM7GRaRPKFns3Q6/urx/z8Yj5VCVo258VKXOxkVkUFKywKBlQ46z8TwNmDobF5ERRMmifCxc+vxARyEiMqgNzt+Vi4jIoFLUZGFmi83sNTPbaGZXdbHeOWbmZlYfTE83s71mtj54/KiYcYqISNeKVg1lZiXAKmAR0Ag8b2ar3X1DxnrVwGXAcxmbeMPd5xUrPhERKVwxryzmAxvd/U133w/cA5yVY72vAd8C9hUxFhER6YViJoupwN9SphuDeUlmdjRwkLs/nKP8DDP7k5n93sxOzPUGZnaxmTWYWUNra2ufBS4iIumKmSxy3fyf7P/BzELAt4H/lWO9TcDB7n408EXgbjMbnbUx91vcvd7d62tqavoobBERyVTMZNEIHJQyPQ1oSpmuBo4CnjKzt4DjgNVmVu/ube6+FcDd1wFvAIcXMVYREelCMZPF88BMM5thZqOAZcDqjoXuvtPdJ7r7dHefDjwLLHH3BjOrCRrIMbNDgJnAm0WMVUREulC0u6HcPWpmlwKPASXAre7+ipldDzS4++ouip8EXG9mUSAGfM7dtxUrVhER6Zq599FQiQOsvr7eGxoaBjoMEZEhxczWuXt9vvX0C24REclrWPcN1d7eTmNjI/v26SccI0VZWRnTpk0jElEnjiJ9aVgni8bGRqqrq5k+fTqmbryHPXdn69atNDY2MmPGjIEOR2RYGdbVUPv27WPChAlKFCOEmTFhwgRdSYoUwbBOFoASxQij/2+R4hj2yUJERHpPyaLIduzYwQ9+8IMelV25ciV79vRwXO4iW7hwIT29Vfmhhx5iw4YDnQ8Xsq19+/Yxf/58PvCBD3DkkUdy7bXX9ui9RaRnlCyKbLgmi97ITBaFKC0t5Xe/+x0vvvgi69ev59FHH+XZZ58tUoQikmlY3w2V6rpfvcKGpnf7dJuz60Zz7d8d2eU6V111FW+88Qbz5s1j0aJF1NbW8otf/IK2tjaWLl3Kddddx+7du/nkJz9JY2MjsViMr3zlKzQ3N9PU1MQpp5zCxIkTefLJJ3Nuv6qqihUrVvDEE08wbtw4vvGNb/ClL32Jt99+m5UrV7JkyRLeeust/uEf/oHdu3cD8P3vf5/jjz+eBx98kFWrVvH444+zefNmTj75ZJ5++mkmT56c9T579+7lwgsvZMOGDcyaNYu9e/cml61Zs4Zrr72WtrY2Dj30UG677TaqqqqYPn06n/rUp5Kx33333bS0tLB69Wp+//vf8/Wvf50HHngAgPvuu49LLrmEHTt28JOf/IQTT0zvaNjMqKqqAhK3RLe3t6t9QqQf6cqiyG644QYOPfRQ1q9fz6JFi3j99ddZu3Yt69evZ926dTz99NM8+uij1NXV8eKLL/Lyyy+zePFiLrvsMurq6njyySc7TRQAu3fvZuHChaxbt47q6mquueYaHn/8cR588EG++tWvAlBbW8vjjz/OCy+8wL333stll10GwNKlS5k8eTKrVq3ioosu4rrrrsuZKAB++MMfUlFRwUsvvcS//Mu/sG7dOgC2bNnC17/+dZ544gleeOEF6uvruemmm5LlRo8ezdq1a7n00ku5/PLLOf7441myZAn/9m//xvr16zn00EMBiEajrF27lpUrV3LdddcB0NTUxMc+9rHktmKxGPPmzaO2tpZFixaxYMGCXvzPiEh3jJgri3xXAP1hzZo1rFmzhqOPPhqAXbt28frrr3PiiSdyxRVXcOWVV3LmmWdmnVV3ZdSoUSxevBiAOXPmUFpaSiQSYc6cObz11ltA4kz80ksvZf369ZSUlPDXv/41Wf573/seRx11FMcddxznnntup+/z9NNPJ5PM3LlzmTt3LgDPPvssGzZs4IQTTgBg//79fOhDH0qW69jmueeeyxe+8IVOt/+JT3wCgGOPPTYZd11dHY888khynZKSEtavX8+OHTtYunQpL7/8MkcddVRB+0lEemfEJIvBwN25+uqr+exnP5u1bN26dTzyyCNcffXVnHbaacmrgnwikUiyOiYUClFaWpp8HY1GAfj2t7/NpEmTePHFF4nH45SVlSXLv/POO4RCIZqbm4nH44RCnV9s5qr2cXcWLVrEz3/+87xluqo26oi7pKQkGXdnxo4dy8KFC3n00UeVLET6iaqhiqy6upr33nsPgNNPP51bb72VXbt2AYkDdUtLC01NTVRUVLB8+XKuuOIKXnjhhayyvbFz506mTJlCKBTizjvvJBaLAYmqnwsvvJC7776bWbNmpVUfZTrppJO46667AHj55Zd56aWXADjuuON45pln2LhxIwB79uxJu3K59957k88dVxw9+Vytra3s2LEDSLSfPPHEExxxxBHd2oaI9JyuLIpswoQJnHDCCRx11FGcccYZnHfeecmDZlVVFT/72c/YuHEj//zP/0woFCISifDDH/4QgIsvvpgzzjiDKVOmdNlukc8ll1zC2WefzX333ccpp5xCZWUlAN/4xjc48cQTOfHEE5k3bx4f/OAH+fjHP86sWbOytvH5z3+eCy+8kLlz5zJv3jzmz58PQE1NDbfffjvnnnsubW1tAHz961/n8MMTY1W1tbWxYMEC4vF48upj2bJlXHTRRXz3u9/l/vvv7zTupqYmPvOZz/DII4+wadMmzj//fGKxGPF4nE9+8pOceeaZPd4nItI9ebsoN7NDgUZ3bzOzhcBc4KfuvqMf4itYri7KX3311ZwHPukf06dPp6GhgYkTJ/br++r/XaRwfdlF+QNAzMwOA34CzADu7mV8IiIyhBRSDRUPRr1bCqx09++Z2Z+KHZikW7BgQbKap8Odd97JnDlz+vR9HnvsMa688sq0eTNmzODBBx/s9rY67moSkaGvkGTRbmbnAucDfxfM02AB/ey5557rl/c5/fTTOf300/vlvURk6CikGupC4EPAv7r7f5nZDOBnxQ1LREQGk7xXFu6+AbgMwMzGAdXufkOxAxMRkcEj75WFmT1lZqPNbDzwInCbmXV+Q76IiAw7hVRDjXH3d4FPALe5+7HAqcUNS0REBpNCkkXYzKYAnwQe7s7GzWyxmb1mZhvN7Kou1jvHzNzM6lPmXR2Ue83MhmyL63Dtory/x7OAxO825syZw7x586ivz3tbuIj0oUKSxfXAY8Ab7v68mR0CvJ6vkJmVAKuAM4DZwLlmNjvHetUk2kSeS5k3G1gGHAksBn4QbG/IGa7Jojd6Mp5FhyeffJL169f3OFGJSM8U0sB9H3BfyvSbwNkFbHs+sDFYHzO7BzgLyDxKfA34FnBFyryzgHvcvQ34LzPbGGzvjwW8b26/uQo2/7nHxXOaPAfO6LqtX+NZ9M14FiIysApp4J5mZg+aWYuZNZvZA2Y2rYBtTwX+ljLdGMxL3fbRwEHunlm9lbdsUP5iM2sws4bW1tYCQup/Gs+i78azMDNOO+00jj32WG655ZZe/K+ISHcV8qO820h07/H3wfTyYN6iPOVy9Ued7IjKzELAt4ELuls2OcP9FuAWSPQN1WU0ea4A+oPGs+jdeBbPPPMMdXV1tLS0sGjRIo444ghOOumkgvaTiPROIcmixt1vS5m+3cwuL6BcI3BQyvQ0oClluho4CngqGOdgMrDazJYUUHZI0ngWvRvPoq6uDkhcKS1dupS1a9cqWYj0k0IauLeY2XIzKwkey4GtBZR7HphpZjPMbBSJBuvVHQvdfae7T3T36e4+HXgWWOLuDcF6y8ysNPjF+ExgbTc/26Cg8Sz6ZjyL3bt3J8vs3r2bNWvWaOAjkX5UyJXFPwLfJ1Fl5MB/kugCpEtB54OXkriTqgS41d1fMbPrgQZ3X91F2VfM7BckGsOjwAp3jxUQ66Cj8Sz6ZjyL5uZmli5dCiSS3HnnnZesfhOR4ss7nkXOQmaXu/vKIsTTYxrPYvDReBYig19fjmeRyxd7WE5ERIagng6r2nlLpRSFxrMQkYHU02TR/bor6RWNZyEiA6nTZGFm75E7KRhQXrSIRERk0Ok0Wbh7dX8GIiIig1dPG7hFRGQEUbIQEZG8lCyKbLh2Ud7f41m89tprzJs3L/kYPXo0K1cOqp/6iAxrShZFNlyTRW/0ZDyL97///axfvz7ZW29FRUXyF90iI11PflzdXXlvne3krqidQAPwvzrGqxjsblx7I3/Z9pc+3eYR44/gyvlXdrmOxrPo+/Esfvvb33LooYfyvve9r+v/IJEhyN3Z3b6b7W3b2bFvR+K5bQfb9+V+3tG2g0PGHMJti2/Lv/FeKOR3FjeR6PH1bhK3zS4j0UPsa8CtwMJiBTcc3HDDDbz88susX7+eNWvWcP/997N27VrcnSVLlvD000/T2tpKXV0dv/71r4FEx39jxozhpptu4sknn+yyu4yO8SxuvPFGli5dmhzPYsOGDZx//vksWbIkOZ5FWVkZr7/+Oueeey4NDQ0sXbqUBx54gFWrVvHoo48WPJ7FSy+9xDHHHAOkj2dRWVnJjTfeyE033ZTsNbdjPIuf/vSnXH755Tz88MMsWbKEM888k3POOSe5/Y7xLB555BGuu+46nnjiibS+oVLdc889XXanLjKY7I3uPXDQ7+rgn7I8Gs/d83LYwowtG8vY0rGMKxvHoWMPZVzpOA4Ze0jRP0chyWKxuy9Imb7FzJ519+vN7MvFCqyv5bsC6A8az6J341l0bH/16tV885vfzLtvRPra/tj+Tg/wqc+py/bF9uXclmGMLR3L2LKxjCsdx0FVBzF34txkIkh7Lh3H2LKxVEWquuzqv5gKSRZxM/sk0NE96Dkpy/RL7m7QeBa9G88C4De/+Q3HHHMMkyZN6nQdkUJE41F2tu0s6ODfcTWwu313p9urHlWdPKjXVtRy+LjDsw/6KQf/6lHVlIRK+vET904hyeLTwHeAjlbaPwLLzawcuLRYgQ0XmeNZfOUrX+HTn/40VVVVvPPOO0QiEaLRKOPHj2f58uVUVVVx++23p5Xtba+tO3fuZNq0aYRCIe64446c41n89Kc/5aabbuKKK67IuY2O8SxOOeWUrPEsVqxYwcaNGznssMPYs2cPjY2NyS7K7733Xq666qpej2fR4ec//7mqoCRL3OO8t/+9Lqt2MquA3t3/bqfbqwhXJA/sY8vGMn3M9C7P+MeUjiESivTjJ+5/eZNF0ID9d50s/kPfhjP8aDyLvhnPAhIDKz3++OPcfPPNPd4XMvj1pIF3R9sO4h7Pub1RoVGMKxuXPMDXVdYlq36ynoPkUFpS2s+fevDLO56FmU0DvgecQKLa6Q/AP7l7Y/HDK5zGsxh8NJ6FQHEbeFPP7js7+JeHywesnn8oKHQ8i0KqoW4jcSfU3wfTy4N5i3oenogMRSO5gXekKyRZ1Lh76g28t5vZ5cUKSHLTeBbS19TAK91RSLLYYmbLgY7bXc4FthYvJMlF41lIV9TAK8VWSLL4R+D7wLdJtFn8J3BhMYMSGcnUwCuDUSF3Q70NLEmdF1RDqRc3kQK0xdrYtndbUX/BqwZeKbaeDqv6RZQsRNgf20/znmY2796cfN68ezPNu5uT09vbtucsqwZeGUp6miwK+ms1s8UkftBXAvyHu9+QsfxzwAogBuwCLnb3DWY2HXiVRP9TAM+6++d6GKtIj7TH22nd03ogAaQkg817Eglh677s5rvRo0YzuXIykyomcdTEo5hcOZmJ5RPVwCtDWk+TRd5uPsysBFhF4hbbRuB5M1vt7ql9U9/t7j8K1l9CotPCxcGyN9x9Xg/jGzR27NjB3XffzSWXXNLtsitXruTiiy+moqKiCJH1zsKFC/n3f/936uvz3p6d5aGHHuLwww9n9uzZ3drWd77zHX784x/j7lx00UVcfnnPb8qLxqNs2bsl7cCfmRC27N2CZ/ypV0WqEomgchKzxs9iUuUkJldMTs6bXDGZisjg+/8S6a1Ok0UnXZND4qqivIBtzwc2dnRhbmb3AGcByWTh7qm3Y1R28n5DWsd4Fj1NFsuXLx+UyaI3HnroIc4888xksijEyy+/zI9//GPWrl2b7Dzx4x//ODNnzsxa1/EDVwR7Nue8Mtiydwsxj6WVKw+XM7lyMpMrJvPhqR9OXh1MrpycfF01qqrXn19kKOo0Wbh7dS+3PRX4W8p0I7AgcyUzW0GiDWQU8JGURTPM7E/Au8A17v5/ehPM5m98g7ZX+3Y8i9JZRzD5y113vKvxLPpmPItXX32V4447jvLycqIe5fgTj+fn9/2cFV9YQXu8nWg8Snu8nfZYO5t3beZT930q/f+qpDSZCBZMWZCVCCZXTqY6Uq32AJFO9LQaqhC5vnVZVw7uvgpYZWbnAdcA5wObgIPdfauZHQs8ZGZHZlyJYGYXAxcDHHzwwX0df5/QeBaFj2fx3HPP8fCvH+ba/+9a/vdv/jdv/+1tLrvkMu64/w7Gvm8sv33qt/xx4x8ZVTqK1Q+v5sgPHMnm3ZsxMyKhCOFQmIpIBZWjKrlmwTWJaqEgQYwpHaNEINILxUwWjcBBKdPTSAyi1Jl7gB8CuHsb0Ba8XmdmbwCHkxidL8ndbwFugUTfUF0Fk+8KoD+M5PEsli1bxhe+8AX2RffRHm9n1/5dNO9uJhqPsi+6jw+e+kFe3fYqow8ZzcY3N/L2u2/DGPje3d9jT3QPh73/MFZ8YQWf/fvPUlVVxTHzjqGyopL3j38/JVaSlgjeG/Uenzoi/cpCRHqnmMnieWCmmc0A3iExwt55qSuY2Ux3fz2Y/DjwejC/Btjm7jEzOwSYCQyJ4Vu7MpzHs/jZXT9LVAMFj5Y9LcQ8xjvvvUNse4y9bXuJE+eNHW+we/9utu/bzpa9W5K/+q2qqGJ82XjC1WEsbhwy5hDCoTDhUDj5vlesuIIrViS6UP/yl7/MtGnTCIeK+ScsIh06PzL0krtHSYx38RiJ22B/4e6vmNn1wZ1PAJea2Stmtp5Eu8X5wfyTgJfM7EUSgy59zt23FSvWYsocz+LWW29l165dQOJA3dLSQlNTExUVFSxfvpwrrriCF154Iatsb+zcuZMpU6YQCoW48847c45nMWvWLG666aac5WPxGMefcDy333k72/dt5/fP/56XXnqJTbs2UXtELU/9n6dY07CGN3a8wWubX+OPL/6R1j2txD3Or375K8rCZTz966eZv2A+06qnMWXCFKq9mtkTZnP4+MMpC5cl2w3Gl4/HzCiPlBMpiaQlqJaWFgDefvttfvnLX2pcC5F+VNTTMnd/BHgkY95XU17/UyflHgAeKGZs/WWwj2fx4Q9/mPnHz+ew2Ydx8vEn86GPfIgZh884cJUQayfucT667KM8edmTLDh2AUccdQRzjplDzGNMmTSF7978Xb78+S+zf/9+DOO6r13HrPmziIQiVIYqOefUc5LjWYwpHcPy85Zz0UUXser7q7o1nsXZZ5/N1q1biUQirFq1inHjxvV4n4hI9+Qdz2Ko0HgW2eIeT7tLqCMBJOfF24nFY1nlSkIlREKR5CMcChMpSZ8OWf6LUo1nITL49eV4FjIIpSaCXAkhXyIIh8KUh8vTk0JJmEgoUlAiEJGRRcliEHL3tLP/9ng7p510Gm1tbbg7juPufPMH3+Tw2Ycny4UslLwCKA+XJ64IMq4Q8nUvofEsRCSXYZ8s3H1Q3V+fmQgyk0J7rD1nj6N3PXpX4qBfEslKAoUmgkIM9fEshku1qshgM6yTRVlZGVu3bmXChAn9kjDcnahHicbSE0BqNVE0Hs3qb6jjR2WRUISqSFWyOij1EbLQoEp6g5G7s3Xr1rRbg0WkbwzrZDFt2jQaGxtpbW3tk+3FPU4sHiPmKY+U6Xg8np0IMEpCJZRYCSELJV+XWEnytZkRDf7tZW8n7y6FKCsrY9q0aQMdhsiwM6yTRSQSYcaMGXnXc3fe3f9udjfUKdPNe5ppi6WPgR0OhZlUMSmrj6HU6XGl43RFICJD3rBOFoVo2dPCmQ+eyd5o+hl9iZVQW1HL5MrJzJ4wm48c/JFkP0MdfQ6NLxuvO4dEZEQY8cliXNk4zjn8nOSYBB1XBhPLJ2pgGhGRwIhPFpFQhC998EsDHYaIyKCmOhQREclLyUJERPJSshARkbyULEREJC8lCxERyUvJQkRE8lKyEBGRvJQsREQkLyULERHJS8lCRETyUrIQEZG8lCxERCSvoiYLM1tsZq+Z2UYzuyrH8s+Z2Z8xUVKIAAAQbklEQVTNbL2Z/cHMZqcsuzoo95qZDd1xPkVEhoGiJQszKwFWAWcAs4FzU5NB4G53n+Pu84BvATcFZWcDy4AjgcXAD4LtiYjIACjmlcV8YKO7v+nu+4F7gLNSV3D3d1MmKyE5JulZwD3u3ubu/wVsDLYnIiIDoJjjWUwF/pYy3QgsyFzJzFYAXwRGAR9JKftsRtmpxQlTRETyKeaVRa6Bpz1rhvsqdz8UuBK4pjtlzexiM2sws4bW1tZeBSsiIp0rZrJoBA5KmZ4GNHWx/j3A/+hOWXe/xd3r3b2+pqaml+GKiEhnipksngdmmtkMMxtFosF6deoKZjYzZfLjwOvB69XAMjMrNbMZwExgbRFjFRGRLhStzcLdo2Z2KfAYUALc6u6vmNn1QIO7rwYuNbNTgXZgO3B+UPYVM/sFsAGIAivcPVasWEVEpGvmntUUMCTV19d7Q0PDQIchIjKkmNk6d6/Pt55+wS0iInkpWYiISF5KFiIikpeShYiI5KVkISIieRWzuw8REekD3t5OdMsWos3NtLe0EG1pJdrSQrS5mWhrC+EpU6j7138tagxKFiIiA8TjcWLbtxNtaaG9uTmRAFpaE0mgpYX21haizS3Etm2DzJ85hMOEa2qI1NYSqqgseqxKFiIifczdie/alTz7b89IAtGWlsS81laIRrPKl0yYQHhSLZGaWsqPPIrwpEmEa2sI19YSmTSJcG0tJePGYaH+a0lQshAR6Yb4vn1EW1PO/lsSZ/8HkkAz0ZZWfO/erLKh6upEEqitpXL+/CAJ1BKuTVwhhCdNIjxxIhaJDMAn65qShYgI4NEo0a1bO00CHfPiO3dmlbXS0sRBf1It5UceSXhhbXoimDSJcE0NoYqKAfhkfUPJQkSGNXcntmNHehVQc/OBRuLgaiC2ZWt2u0BJCeGJEwlPmkTkfQdT8cF6wrUdSaCWyKTEc2j0aMxyjawwfChZiMiQFdu1m2hLc/rZf+rVQHMz0dZWvL09q2zJuHHJtoDSWUcQSUkCHYmgZPx4rEQjOoOShYgMQvH9+4Mz/06uBoKrhPiePVllQ5WVySqg8vpjgyqgjCRQU0No1KgB+GRDl5KFiPQbj8WCdoEWoq2pSSD9bqHYjh1ZZS0SSSaB0iOOoPKkE5N3BoVrEu0F4ZpaSqqKfxvpSKRkISK95u7Ed+5MrwJqyXHL6JYtEI+nFw6FCE+YkGgXmDaN8mOOPpAEamuDNoIaSsaOHfbtAoOZkoWIdCm+Z09Gg3Duu4V8//6ssiVjxhy4Gjj88ORtoweSQC3hCeOxsA5Fg53+h0RGKN+/n2hra/rZf2tLViKI79qVVdYqKpIH/fJ58zKSQHDbaE0NodLSAfhkUgxKFiLDjMfjxLZuDZJAeiJITQKxbduyC0cihGsmEqmdROlhh1F5/PHpiSC4SghVVqpKaIRRshAZItyd+Hvv5e4+ojXlltEtW7K7kDCjZMIEIrW1RCZPpnzu3ESDcG3tgV8O19Ym2gX6sQsJGTqULEQGgfjevel9BuXoPiLa0oLv25dVNjR6dOLHYTW1lB5ySM5+hMITJgzKLiRk6FCyECkij8WS/Qhldx9x4Aoh/u67WWWtrOxAZ3JHHZW7H6GaGkLl5QPwyWSkUbIQ6YX4/v1EN22ivakp8Xin6cDrpibam5sh89fDQdfS4doaSmfMoHL+gux+hGprCVVXq11ABo2iJgszWwx8BygB/sPdb8hY/kXgM0AUaAX+0d3/O1gWA/4crPq2uy8pZqwiucR27UokgE2Jg380IylEW1vTC5glqn/q6ij/wAcYXVdHpG4K4cmTD3QtPX682gVkyClasjCzEmAVsAhoBJ43s9XuviFltT8B9e6+x8w+D3wL+FSwbK+7zytWfCLuTmzbttxXBMEjs3rIIhHCU6YQqauj8sQTidTVHXhMrSMyaRKmbiRkGCrmlcV8YKO7vwlgZvcAZwHJZOHuT6as/yywvIjxyAjj0WiigTg1AaQmhU2bshqMQ5WVyYN/xTFHE6mrSyaHSN1UwjUTdVUgI1Ixk8VU4G8p043Agi7W/5/Ab1Kmy8ysgUQV1Q3u/lDfhyhDWXzfPtqbOtoL3kkmgWjHvOZmiMXSypSMH0+kro7SmTOpOvnkA1cEQYIYCV1Ni/REMZNFrm+c55iHmS0H6oGTU2Yf7O5NZnYI8Dsz+7O7v5FR7mLgYoCDDz64b6KWQSP27rudNxw3NRHbujW9QCiU6F+oro7yY48N2gtSqoimTNGdQyI9VMxk0QgclDI9DWjKXMnMTgX+BTjZ3ds65rt7U/D8ppk9BRwNpCULd78FuAWgvr4+ZyKSwSn5K+POqoiamrK6mbDSUiJBlVDZR05Jby+oqyM8aZL6GBIpkmJ+s54HZprZDOAdYBlwXuoKZnY0cDOw2N1bUuaPA/a4e5uZTQROINH4LUOEt7fT3tyckgDeSUsE0U2bszqeC1VXB1cBU6n44AezqohKJkxQFZHIAClasnD3qJldCjxG4tbZW939FTO7Hmhw99XAvwFVwH3BQaDjFtlZwM1mFgdCJNosNuR8IxkQ8b17u6wiira0ZHVFXVIzMXFVMHs2kVNPTSSBKQcSQkl19QB9GhHJxzxzzNkhqr6+3hsaGgY6jGGhY8ziA1cBm7ISQmz79vRC4TCRoL0g84qg444i9UAqMviY2Tp3r8+3nip4RyCPxxNdU6clgPS7iTKHq7SysuSBv+zII7MSQri2VmMViwxjShbDkO/fT/vmzZ3/2Gzz5qwuKErGjCE8tY5R06dTefzxKVcFU4lMrdMoZSIjnJLFEBTbtTvjSqAp5fcGQRcUqdWLZoRrahK3lM6Zw+jFp2fdSRSq1LjFItI5JYtBxt2Jbd/eafcT7U1NxHfuTC8UiSRvKa388IczEsEUIpMnqwsKEekVJYt+5rFYehcUmUlh0yZ87960MqGKCiJT6wjX1VE+7wMZyUBdUIhI8SlZ9LF4W1v2baQdSWHTpkQXFBmjmCW7oDjsMKpOPDHrTqLQmDFqLxCRAaVk0U2x997rspfS2JYt6QVSu6A45hh1QSEiQ5KSRQp3J7ZlS8rBf1N2e8F776WVsVGjEu0FU+soXXhyVhVRZFKthrMUkSFvxCeL6JYtNH3pS8lqok67oKiro6K+Pv33BVOmJLqgUHuBiAxzIz5ZhKqqiO3aTemsWVR99KNZv0BWFxQiIkoWhMrKmPGLewc6DBGRQU31JyIikpeShYiI5KVkISIieSlZiIhIXkoWIiKSl5KFiIjkpWQhIiJ5KVmIiEhew2YMbjNrBf67F5uYCGzJu1b/U1zdo7i6R3F1z3CM633uXpNvpWGTLHrLzBoKGbS8vymu7lFc3aO4umckx6VqKBERyUvJQkRE8lKyOOCWgQ6gE4qrexRX9yiu7hmxcanNQkRE8tKVhYiI5DWikoWZ3WpmLWb2cifLzcy+a2YbzewlMztmkMS10Mx2mtn64PHVforrIDN70sxeNbNXzOyfcqzT7/uswLj6fZ+ZWZmZrTWzF4O4rsuxTqmZ3Rvsr+fMbPogiesCM2tN2V+fKXZcKe9dYmZ/MrOHcyzr9/1VQEwDua/eMrM/B+/bkGN58b6P7j5iHsBJwDHAy50s/xjwG8CA44DnBklcC4GHB2B/TQGOCV5XA38FZg/0Piswrn7fZ8E+qApeR4DngOMy1rkE+FHwehlw7yCJ6wLg+/39Nxa89xeBu3P9fw3E/iogpoHcV28BE7tYXrTv44i6snD3p4FtXaxyFvBTT3gWGGtmUwZBXAPC3Te5+wvB6/eAV4GpGav1+z4rMK5+F+yDXcFkJHhkNgqeBdwRvL4f+KiZ2SCIa0CY2TTg48B/dLJKv++vAmIazIr2fRxRyaIAU4G/pUw3MggOQoEPBdUIvzGzI/v7zYPL/6NJnJWmGtB91kVcMAD7LKi+WA+0AI+7e6f7y92jwE5gwiCIC+DsoOrifjM7qNgxBVYCXwLinSwfiP2VLyYYmH0FiSS/xszWmdnFOZYX7fuoZJEu1xnLYDgDe4HET/I/AHwPeKg/39zMqoAHgMvd/d3MxTmK9Ms+yxPXgOwzd4+5+zxgGjDfzI7KWGVA9lcBcf0KmO7uc4EnOHA2XzRmdibQ4u7rulotx7yi7a8CY+r3fZXiBHc/BjgDWGFmJ2UsL9r+UrJI1wikniVMA5oGKJYkd3+3oxrB3R8BImY2sT/e28wiJA7Id7n7L3OsMiD7LF9cA7nPgvfcATwFLM5YlNxfZhYGxtCPVZCdxeXuW929LZj8MXBsP4RzArDEzN4C7gE+YmY/y1inv/dX3pgGaF91vHdT8NwCPAjMz1ilaN9HJYt0q4H/J7ij4Dhgp7tvGuigzGxyRz2tmc0n8f+2tR/e14CfAK+6+02drNbv+6yQuAZin5lZjZmNDV6XA6cCf8lYbTVwfvD6HOB3HrRMDmRcGfXaS0i0AxWVu1/t7tPcfTqJxuvfufvyjNX6dX8VEtNA7KvgfSvNrLrjNXAakHkHZdG+j+G+2MhQYWY/J3GXzEQzawSuJdHYh7v/CHiExN0EG4E9wIWDJK5zgM+bWRTYCywr9gEmcALwD8Cfg/pugC8DB6fENhD7rJC4BmKfTQHuMLMSEsnpF+7+sJldDzS4+2oSSe5OM9tI4gx5WZFjKjSuy8xsCRAN4rqgH+LKaRDsr3wxDdS+mgQ8GJwDhYG73f1RM/scFP/7qF9wi4hIXqqGEhGRvJQsREQkLyULERHJS8lCRETyUrIQEZG8lCxERCQvJQuRfhZ0M92jX5MH3WPX9cW2RLpDyUJkaLkAqMu3kkhfU7KQEcvMppvZX8zsP8zsZTO7y8xONbNnzOx1M5sfPP7TEgPh/KeZvT8o+0UzuzV4PScoX9HJ+0wwszXBNm4mpbM3M1tuiYGJ1pvZzcGvrDGzXWb2/5vZC2b226DLjnOAeuCuYP3yYDP/b7Den83siGLuMxm5lCxkpDsM+A4wFzgCOA/4MHAFiS5E/gKc5O5HA18FvhGUWwkcZmZLgduAz7r7nk7e41rgD8E2VhN0S2Jms4BPkehJdB4QAz4dlKkEXgh6GP09cK273w80AJ9293nuvjdYd0uw3g+DuEX63IjqG0okh/9y9z8DmNkrwG/d3c3sz8B0Er2c3mFmM0l09dzRZ1fczC4AXgJudvdnuniPk4BPBOV+bWbbg/kfJdFj6fNBfz/lJMabgMRYCvcGr38G5Orxt0PHsnUd7yPS15QsZKRrS3kdT5mOk/h+fA140t2XWmKgpadS1p8J7KKwNoRcnbAZcIe7X93D8h06Yo6h77QUiaqhRLo2BngneH1Bx0wzG0Oi+uokYELQntCZpwmql8zsDGBcMP+3wDlmVhssG29m7wuWhUj0nAuJqrE/BK/fIzHuuEi/UrIQ6dq3gG+a2TNAScr8bwM/cPe/Av8TuKHjoJ/DdcBJZvYCiTEI3gZw9w3ANSSGyXwJeJxEd+IAu4EjzWwd8BHg+mD+7cCPMhq4RYpOXZSLDEJmtsvdqwY6DpEOurIQEZG8dGUh0kfM7ELgnzJmP+PuKwYiHpG+pGQhIiJ5qRpKRETyUrIQEZG8lCxERCQvJQsREclLyUJERPL6vxgaJuvjwP05AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x24180874710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch2_1.best_score_, gsearch2_1.best_params_))\n",
    "test_means = gsearch2_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch2_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch2_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch2_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch2_1.cv_results_).to_csv('my_preds_maxdepth_min_child_weights_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(max_depth), len(min_child_weight))\n",
    "train_scores = np.array(train_means).reshape(len(max_depth), len(min_child_weight))\n",
    "\n",
    "for i, value in enumerate(max_depth):\n",
    "    pyplot.plot(min_child_weight, -test_scores[i], label= 'test_max_depth:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'max_depth' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig('max_depth_vs_min_child_weght_1.png' )"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处使用GridSearchCV实现了对max_depth和min_child_weight参数的调整，并且得到了最佳参数为9和1。\n",
    "我们继续使用GridSearchCV对max_depth和min_child_weight参数进行微调。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': [8, 9, 8], 'min_child_weight': [0, 1, 6]}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_depth = [8,9,8]\n",
    "min_child_weight = [0,1,6]\n",
    "param_test2_2 = dict(max_depth=max_depth, min_child_weight=min_child_weight)\n",
    "param_test2_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.25200, std: 0.00390, params: {'max_depth': 8, 'min_child_weight': 0},\n",
       "  mean: -0.26670, std: 0.00352, params: {'max_depth': 8, 'min_child_weight': 1},\n",
       "  mean: -0.33438, std: 0.00336, params: {'max_depth': 8, 'min_child_weight': 6},\n",
       "  mean: -0.21694, std: 0.00409, params: {'max_depth': 9, 'min_child_weight': 0},\n",
       "  mean: -0.22746, std: 0.00464, params: {'max_depth': 9, 'min_child_weight': 1},\n",
       "  mean: -0.29274, std: 0.00290, params: {'max_depth': 9, 'min_child_weight': 6},\n",
       "  mean: -0.25200, std: 0.00390, params: {'max_depth': 8, 'min_child_weight': 0},\n",
       "  mean: -0.26670, std: 0.00352, params: {'max_depth': 8, 'min_child_weight': 1},\n",
       "  mean: -0.33438, std: 0.00336, params: {'max_depth': 8, 'min_child_weight': 6}],\n",
       " {'max_depth': 9, 'min_child_weight': 0},\n",
       " -0.21694215121365162)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_2 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #第一轮参数调整得到的n_estimators最优值\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch2_2 = GridSearchCV(xgb2_2, param_grid = param_test2_2, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch2_2.fit(X_train , y_train)\n",
    "\n",
    "gsearch2_2.grid_scores_, gsearch2_2.best_params_,     gsearch2_2.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([4231.9591496 , 4220.6499805 , 5577.19362473, 6370.40666432,\n",
       "        5378.30063534, 4446.20112257, 4093.18652415, 4028.96007681,\n",
       "        3664.39089193]),\n",
       " 'mean_score_time': array([54.23289852, 50.2219244 , 67.20338697, 66.69234853, 46.10640655,\n",
       "        45.35672684, 41.05684776, 38.80188503, 29.16410332]),\n",
       " 'mean_test_score': array([-0.25200334, -0.26669637, -0.33437901, -0.21694215, -0.22746497,\n",
       "        -0.29274355, -0.25200334, -0.26669637, -0.33437901]),\n",
       " 'mean_train_score': array([-0.08233311, -0.09972232, -0.17019217, -0.04244766, -0.05790876,\n",
       "        -0.12387251, -0.08233311, -0.09972232, -0.17019217]),\n",
       " 'param_max_depth': masked_array(data=[8, 8, 8, 9, 9, 9, 8, 8, 8],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_min_child_weight': masked_array(data=[0, 1, 6, 0, 1, 6, 0, 1, 6],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'max_depth': 8, 'min_child_weight': 0},\n",
       "  {'max_depth': 8, 'min_child_weight': 1},\n",
       "  {'max_depth': 8, 'min_child_weight': 6},\n",
       "  {'max_depth': 9, 'min_child_weight': 0},\n",
       "  {'max_depth': 9, 'min_child_weight': 1},\n",
       "  {'max_depth': 9, 'min_child_weight': 6},\n",
       "  {'max_depth': 8, 'min_child_weight': 0},\n",
       "  {'max_depth': 8, 'min_child_weight': 1},\n",
       "  {'max_depth': 8, 'min_child_weight': 6}],\n",
       " 'rank_test_score': array([3, 5, 8, 1, 2, 7, 3, 5, 8]),\n",
       " 'split0_test_score': array([-0.25200872, -0.26609389, -0.33152376, -0.21653047, -0.22518801,\n",
       "        -0.29092523, -0.25200872, -0.26609389, -0.33152376]),\n",
       " 'split0_train_score': array([-0.08339853, -0.10054927, -0.17035189, -0.0422514 , -0.05766512,\n",
       "        -0.12264549, -0.08339853, -0.10054927, -0.17035189]),\n",
       " 'split1_test_score': array([-0.25865938, -0.27320994, -0.34061711, -0.22384677, -0.23653574,\n",
       "        -0.29776069, -0.25865938, -0.27320994, -0.34061711]),\n",
       " 'split1_train_score': array([-0.08261367, -0.10023146, -0.17075394, -0.04252833, -0.05829862,\n",
       "        -0.12382511, -0.08261367, -0.10023146, -0.17075394]),\n",
       " 'split2_test_score': array([-0.25166075, -0.26596803, -0.33440973, -0.21773338, -0.22577751,\n",
       "        -0.29422834, -0.25166075, -0.26596803, -0.33440973]),\n",
       " 'split2_train_score': array([-0.08140762, -0.09927293, -0.16898904, -0.04220461, -0.05715124,\n",
       "        -0.12373224, -0.08140762, -0.09927293, -0.16898904]),\n",
       " 'split3_test_score': array([-0.25125217, -0.26572573, -0.33401505, -0.21538858, -0.22637482,\n",
       "        -0.29082052, -0.25125217, -0.26572573, -0.33401505]),\n",
       " 'split3_train_score': array([-0.08139177, -0.0987457 , -0.1693909 , -0.04257142, -0.05789483,\n",
       "        -0.12429362, -0.08139177, -0.0987457 , -0.1693909 ]),\n",
       " 'split4_test_score': array([-0.24643474, -0.26248352, -0.33132892, -0.21121063, -0.22344803,\n",
       "        -0.28998253, -0.24643474, -0.26248352, -0.33132892]),\n",
       " 'split4_train_score': array([-0.08285395, -0.09981224, -0.17147508, -0.04268255, -0.05853398,\n",
       "        -0.1248661 , -0.08285395, -0.09981224, -0.17147508]),\n",
       " 'std_fit_time': array([ 56.8946506 , 215.96476022, 856.81521731, 428.7707841 ,\n",
       "        446.25639264,  18.26796311,  14.06316935,  21.59755166,\n",
       "        602.79868407]),\n",
       " 'std_score_time': array([ 7.3420591 , 12.43403217, 15.43403013,  4.43355638,  1.4217008 ,\n",
       "         2.61751986,  2.42453518,  1.31369479,  8.5689597 ]),\n",
       " 'std_test_score': array([0.00389853, 0.00352167, 0.00336167, 0.00409224, 0.00463979,\n",
       "        0.00289843, 0.00389853, 0.00352167, 0.00336167]),\n",
       " 'std_train_score': array([0.00080346, 0.00064914, 0.00090294, 0.00018686, 0.00048503,\n",
       "        0.00073377, 0.00080346, 0.00064914, 0.00090294])}"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch2_2.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.216942 using {'max_depth': 9, 'min_child_weight': 0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAELCAYAAAD+9XA2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xl4VOXZ+PHvM5ONkH0DYoCwyb4HARVEAQVFtrKoVaFKcS1qf22lb32rvq3viy1VXFotrXVrK4moCLIpm6hlMSCEfYcQErLve2ae3x/nJISQZbJOJrk/15UrmTlnztwnhNx57vOc+1Faa4QQQojGsjg7ACGEEG2DJBQhhBBNQhKKEEKIJiEJRQghRJOQhCKEEKJJSEIRQgjRJCShCCGEaBKSUIQQQjQJSShCCCGahJuzA2hJISEhOjIy0tlhCCGES9m3b1+a1jq0rv3aVUKJjIwkNjbW2WEIIYRLUUpdcGQ/KXkJIYRoEpJQhBBCNAlJKEIIIZqEJBQhhBBNQhKKEEKIJiEJRQghRJOQhCKEEKJJtKv7UBrsxCbISwYvP/D0Ay9/87P52L0DKOXsKIUQwqkkoThi70o4s7Xm7RY3I7F4+ppJxr9S8qnhc9XnPHwkKQkhXJokFEfMex8Ks6A4F4pzoCjH/Jxd5bH5uTgXsi5CcfaV57S99vdQFiMhVSQj31oSUqWEVZHEzA+LVDGFEM7hlISilAoCooFI4DwwT2udWWWfYcBbgB9gA17SWkeb2/4FRAGlwF7gEa11abMF7OlrfDSU1lCSX0Pyqea58s+5SZB2wkhQRTlgd+AUPXyrGQ1Vfa6WEZSnH1jl7wwhRP0prXXLv6lSfwAytNbLlFJLgUCt9bNV9rke0FrrU0qpcGAf0F9rnaWUuhPYaO76b2Cn1vqtut43KipKu2wvL62hrKhK0sm+elRU07bKn23Fdb+Xu3f9ynVXfTavL7l5NP/3RAjRIpRS+7TWUXXt56w/RWcAE8yv3wd2AFclFK31yUpfJyqlUoBQIEtrvaF8m1JqLxDRzPE6n1LGxX/3DuDbqeHHKSs2k08N5bqaRk7ZCVcelxbU/T5Wz7rLddeU9qpsc/OS60pCuBBnJZROWuskAK11klIqrLadlVI3AB7AmSrPuwMPAE/V8trFwGKAbt26NTLsNsDN0/joGNLwY9hKKyWlqteVahgdFedCWsqV50py634fi3uVsp1//UdOHh0lKQnRQpotoSiltgCdq9n0m3oepwvwIbBA62uubP8Fo9z1TU2v11qvBFaCUfKqz3uLGljdwTvI+Ggou+3aZFRRtquhXFecA5nnzdeZyazOyQ7Wambf1TThoUrCKn+dh69MdhDCAc2WULTWk2rappRKVkp1MUcnXYCUGvbzA9YDz2mtd1fZ9jxGCeyRJgxbtBSLFToEGB8NpTWU5FWTfLJruK5kfs65BEXHrjzWtjreSNWchK6a8FDLCMrDVyY7iDbPWT/ha4EFwDLz8+dVd1BKeQCfAR9orT+usm0RcAcwsZpRi2gvVPkvel/guoYdQ2soLXRgOnju1dvyUiD99JV9bCV1v5d7xzruT/KvZkZeletLVveGnacQLcBZCWUZEKOUehiIB+YCKKWigEe11ouAecB4IFgptdB83UKt9QHgbeACsEsZ9fFPtdb/07KnINoEpcDD2/jwra5C66DSokqjotomPGRfSVBFWZAVf2VbWWHd7+PWoe6kU9eEB3evhp+nELVwyrRhZ3HpacOi7bOVXpt0HJ3wUP51SV7d72P1qCbp1HPCg7u3THZoR1r7tGEhRFVWd+gYbHw0lN1Wv+ng5Z8zzl6doKjjD02LWw0jIEcmPJj7ePjIZIc2RhKKEG2JxQodAo2PhrLbjZFOtcmncjmvygiqvu2GUHWMihyY8ODpa5yzaBUkoQghrmaxGL+wvfzAv4HHqGg3lFt7ue6qsl025F2GtJNXtjnUbsinju4NtdynVJ6oZAZek5DvohCi6SkFnj7GB10adoxq2w3VUrYrT1gF6cb9SuXbyorqfq/ydkPVTniobnRUzbUnN8+GnWcbIglFCNE6NVm7oZLqp4NfdZ9SNTPzci5dKe2V5tf9Pte0G3JkwkOVqeIuvraSJBQhRNvm5gFuIY1sN1RW9+iouhtp887Us92QW/Wjo/pMFXfi2kqSUIQQoi5Wt6ZpN1RtZ4fqri9VuvaUdeHqpFXvtZX8IKgHTFvR7F3AJaEIIURLsFiNEYdXQ2c6cG27oer63+WlwOFPIDsess3X2cuMDyShCCGEgJrbDdltcP5biIuGo2uN8prfdTB4LgyZD50GtEh4klCEEMJVJR8xkkjcx5CbaDQhHTADhs6H7je3+I2jklCEEMKV5CTBoY8hLgaSDxkX8ntPgjtegr5TjZliTiIJRQghWrviXDi2zhiNnP0a0HBdFEz9Iwya3bgZbE1IEooQQrRGtjI4s81IIsfXG92oAyPhll8Z10WCezk7wmtIQhFCiNZCa0j8wUgihz+B/FSjL9uw+2DoPRAxqlXf+CgJRQghnC3zAhyKgYPRkH7KuOu+7xRjJNJ7crPfP9JUJKEIIYQzFGbCkTXGaCR+l/Fc95vgxp8ZM7Uaszy2k0hCEUKIllJWDKe+NJLIyc3G0tEhfWHib417RgK6OTvCRpGEIoQQzUlriN9tJJEjnxlLP3cMg1GLjJJWl6Gt+rpIfUhCEUKI5pB2yrzpMMbox+XuDf2mGUmk54Q2uQZL2zsjIYRwlrxUY3ZWXDQk7jcaNfacALf+l5FMPH2cHWGzkoQihBCNUVIAJzYYSeT0VtA26DwEbn8JBv0I/Bq4wJgLkoQihBD1ZbfB+W+Mab7H1hodgP0i4KYlRkkrrL+zI3QKSShCCOGoy4eNkcih1UYzRk8/GDjLSCLdb2rxZoytjSQUIYSoTU5ipWaMh81mjJNhyv/C9VOc2oyxtXFKQlFKBQHRQCRwHpintc6sss8w4C3AD7ABL2mto6vs8wbwE611277SJYRoWcW5xroicdFwbiegjbYndy6HgbOhY7CzI2yVnDVCWQps1VovU0otNR8/W2WfAuBBrfUppVQ4sE8ptVlrnQWglIoCXO9WUiFE62QrhTPbIW4VHN9gNmPsAbc8C0PmtcpmjK2NsxLKDGCC+fX7wA6qJBSt9clKXycqpVKAUCBLKWUF/gjcB8xqgXiFEG2R1sb03oNmM8aCNKMZ4/AfG9dFWnkzxtbGWQmlk9Y6CUBrnaSUCqttZ6XUDRiLIZ8xn3oSWGu+tnkjFUK0PZnnjVUO46o2Y7zHWKzKRZoxtjbNllCUUluAztVs+k09j9MF+BBYoLW2m+WvuVwZ4dT1+sXAYoBu3Vy7T44QohEKMuDoGmM0cnG38Vz3m42pvv2nu2Qzxtam2RKK1npSTduUUslKqS7mCKMLkFLDfn7AeuA5rbX5E8BwoDdw2hydeCulTmute9cQx0pgJUBUVJRu8AkJIVxPWbHRhDEu2mjKaCuB0H4w8XmzGWNXZ0fYpjir5LUWWAAsMz9/XnUHpZQH8Bnwgdb64/LntdbrqTTyUUrl1ZRMhBDtkN1ujEAqmjFmm80YfwpD5xt3sUupvFk4K6EsA2KUUg8D8RglrPKZW49qrRcB84DxQLBSaqH5uoVa6wNOiFcI0dqlnjRvOoyBrHijGWP/u42L6z1uaZPNGFsbpXX7qQJFRUXp2NhYZ4chhGgqFc0YVxlL5yoL9LzVSCL97mrzzRhbilJqn9Y6qq79JGULIVxLeTPGg6vgzLYrzRjv+F+jGaNvdXOBREuQhCKEaP3sNuOO9biYKs0YnzJuOmynzRhbG0koQojW6/KhSs0Yk640Yxx6D3S7sd03Y2xtJKEIIVqX7EtXmjGmHDGaMfa5HYb8H1w/Fdy9nB2hqIEkFCGE8xXlGKWsuGg49w1GM8YbpBmji5GEIoRwDlupcVH94CrjIntZEQT1hAlLjesiQT2dHaGoJ0koQoiWozVc2m9M8z38CRSkQ4cgGP6A2YwxSm46dGGSUIQQzS/jnHldJBrSTxvNGPvdaSSRXhOlGWMbIQlFCNE8CjKM1idxMVeaMUaOg5uehgHTwcvfufGJJicJRQjRdMqK4eQmI4mc3Az2UmnG2I5IQhFCNI7dDvG7jHLW0TVGM0afTjD6EaOk1XmwXBdpJyShCCEaJvWkcXE97mPIjgf3jmYzxnnQcwJYrM6OULQwSShCCMflpRh3rcdFQ9KBK80YJ/630YzRo6OzIxROJAlFCFG7knw4vsEYjZzZbjRj7DIM7vg/sxljJ2dHKFoJSSgO+PPudaTkZ9DJJ5jrfEOJ8A+hZ2AnAr2lNbZoo+w2OPe12YxxndGM0b8r3Pw0DJ4HYf2cHaFohSShOOBfx/5JruXoNc9ruzsWuy/uyhcviy8drf74egQQ6BVISIcgwryDCfcLoat/KN0DOhHuG4BFmtmJ1krrq5sx5l0GT38YNBuG3APdxkozRlErSSgO+HTO25zNSOZidiqJuWkk56eRXphFZnEGOSVZ5JdlU2TPIdeWSGJZHqqoBLKuPY7WFpS9I27aF0+LH95Wf3zd/fH3DCSkQyCh3iF08Qkmwj/UTEIheLnLDV+imWVfMlY5jIuBlKNgcTeaMQ6dD33ukGaMwmGSUBzQ2TeQzr6BgGPD/KzCfM5lphCflUJiXjrJeWmk5KeTWZRJdkkmeWU5FNmyySg9R2pZLpQUQu61x9FaoewdsGofPJQfXlY/IwF5BBDoZYyAwnyCifANoWtAGD0Cw/D38m7akxdtU3kzxoOr4Py3gIauo+GuPxnNGL2DnB2hcEGSUJpBQIeODO/Qg+HhPRzav6C0mItZ6cRnpZCQk0pSXhopBRlkFGaSXZxJbmkWBbZscssuk2k7hS7JR+XbIf3aY2m7JxZ7RzyUL14WPzq6BeDr4U+gZxChHYII8wkm3DeECL8QIgPD6OwjZbh2w1YKp7caF9dPbKzUjPHXMGSuNGMUjSYJpRXwdvekb2g4fUPDHdrfbrdzKSeT85nJJOQYZbiU/AzSCjPIKs40ynC2bArt2eQUJ3CpNA9VWFp9Gc5uxaJ9cMMXT+WLt5s/vu4BBHgGENwhiFDvILr4lF8HCqOrfwgebvJj4zK0hkv7jJHIkU+NZozewTDiQeOmw+tGyk2HosnIbwYXZLFY6BoQTNcAx9eISC/I5VxGMgk5aVzKSeVyXjpphRlkFGWQU5JNXlkWRbYc0krOkFKaC8VFkHPtca6U4XzxUH54W/3wcQ/AzyOAIK9AwjoG09knmOv8QunqH0KPwE74enZowrMXDsk4a9xwGBcNGWfAzQv6TjUurveeCFZ3Z0co2iBJKO1EsLcvwd6+RNHbof3zi4uJz07lQlYKl3JSScpLJ7Ugg4zCDLJKMskrzaLAlkN2WSLpthNmGU7XWIaz2n1wV37GbDg3f/w9Aitmw3XqaJThuvqHERkURqi3r5ThGqIgwxiFHIyGhL2Agsib4eZnpBmjaBGSUES1Onp60j8sgv5hEQ7tX2azkZCdzvmsFC7lppGYk0ZKQTrphRlkFmWSV5pNflk2BfZMckrizTJcGWReeyxtd8OijdlwXha/a8pwnToG08U3hAi/ULoFhNLNPwQ3aztt81FadKUZ46kvzWaM/WHSC0YzRn/H/v2EaAqSUESTcLNaiQwyRhiOsNvtpBfkcT4rhfjsFBLNMlx6UaZRhis2pmMX2nPIL0khuSyvjjJcR9y0Dx4WPzpY/PBx9yfAM5AgryDCOgbTqWMw1/mF0D0gjMiAMDp6ejbxd6AFVTRjXAVHPofibPDpbDRjHHoPdBok10WEU0hCEU5hsVgI9fEj1MePURGOleFyiwu5kJnKhewUEnPSScpLNcpwRRlkF2eRV2pMRMgqSyDddozzJQWoPA1p1RzM5oVF++CufOlg9cfHzR+/SjeldvYJIdw3hG7mZIRgbx/nl+FSTxgX1w99DNkXjWaMA6YbzRh73CLNGIXTOSWhKKWCgGggEjgPzNNaZ1bZZxjwFuAH2ICXtNbR5jYF/B6Ya257S2v9ekvFL5zD17MDgzp3Y1Dnbg7tX1JWRkJOOheyUkjITiUpN43kgnQyCjPJLJ+OXZZNvi2NbNs5LpbkoQps1R5L292w2I0E5Gnxp6PVD193IwGVl+E6+wbT1S+UyMAwwn2DmqYMl5tsLJUbtwqSDoKyQq/bjPVF+t0pzRhFq+KsEcpSYKvWeplSaqn5+Nkq+xQAD2qtTymlwoF9SqnNWussYCHQFeintbYrpRyrs4h2xcPNjZ5BnegZ5FjzQrvdTkp+NuczU4jPTuVybjqX89NIM68DVe6KkG9LIqksD1VcDNnXHutKVwQfPCy+ZleEAPw9AgnqEEiYOR37Ov9QugcY14K83c0yXEk+HF9vzNA6sw203WjGOGWZ0YzRR37cReuktNYt/6ZKnQAmaK2TlFJdgB1a6751vOYgMMdMMHuB+7TWp+vzvlFRUTo2NrbhgQtRRXZRAReyUriQmUpirnEdKKUgncyiLKMrQmkWhbYcSnQONpWHthSiVPX/56w2DwLtmnBbPsG2MrzxxObZE3vAMHwC+tDJp3w2XKg0JxUtSim1T2sdVdd+zhqhdNJaJwGYSaXWP7mUUjcAHsAZ86lewHyl1CwgFViitT7VnAELUR1/L2+GdI5kSOdIh/YvKi3hYvaVMlxOykHsyd9hyTlCHrmkunkQ7x7E925e5KtS7JaLqLwLkHftsa40J/UxuiJUak4a7GVOx5bmpKIFNVtCUUptATpXs+k39TxOF+BDYIHW2m4+7QkUaa2jlFKzgX8A42p4/WJgMUC3bo7V3oVoLl7uHvRxt9Enaasx1Tf1mNGM8fo7jIvrVZox2u12EnOziM9Kuao5qdEVIavm5qTVcLQ56XV+IXQLCJPmpKLeWnXJSynlB+wA/k9r/XGl548DU7TW580L9Fla6zrv2pKSl3Caomw4+rmRRCqaMY4xksjAWU3ajDGzII/zWanEZ6VwKTeVlPyMKs1Jsymy5VCic7GpXLAWVnucq5uT+uJVnoDM5qSh3kF08gmR5qTtQLOUvJRSgUBXrXVcgyMzrAUWAMvMz59X814ewGfAB5WTiWkNcBvGyOQW4GQj4xGi6ZWVwJmtxlTfExvBVgxBveDW/zJuOgxyrHlofQV6+xDo7VOv5qTxWalcyEolMSetojlpeoGxPEP9mpN6YLH7SHPSdqrOEYpSagcwHSP5HMC4ZvG11vrnDX5TpYKBGKAbEA/M1VpnKKWigEe11ouUUvcD7wJHKr10odb6gFIqAPiX+fo88zUH63pfGaGIZqc1JMQaM7QOfwKFGUYzxkFzzGaMI1z+psMym42k3KyrmpMm56eTXph5VXPSYns2pToPuyUPZSmt9ljSnNQ1ODpCcSSh/KC1Hq6UWoQxOnleKRWntR7SVMG2FEkootmknzFuOIyLNhozunlB3zuNO9d73dbumzGWNyeNN+8Huro5adZVZTi7ygVrUbXHqa45aUd3ozdc5eakXXxC6B4YKs1Jm0hTlrzczOsc86jnBXUh2rT8dKMZY1zMlWaMPcbBuF9A/7vBy8/ZEbYaFc1JHeyKUF1z0vLrQFc3J71Uj+ak5WW4K81JgzsE0rljiDQnbSKOJJT/ATYD32qtv1dK9QRkiq5on0qL4OTGSs0YyyBsAEx60WzGeJ2zI2wTGtOcNCEnlaTc9KuakxpdEXIa3JzUx92fQM/Aq5qTlveGa9fNSatwyiwvZ5GSl2gQux0ufGeUs45+DsU54NsFBpvXRToPdnaEop7sdjupBbnEZ6Ve1Zw0rTCDzOLMq5qTluoc7JZ8sNRWhrvSFaGDxf+q5qQh3oF08Ql16eakTVbyUkr9AaNvViGwCRgKPK21/mejoxSiNUs5bvTQivsYchLAwwf6lzdjHC/NGF2YxWKhk48/nXz8G9SctHyRuqubk2ZRaM+5tjlpdexeFb3hypuT+nr4GwmoUnPSrv6hRLaW5qQOcKTkdbvW+lfmXekJGA0ZtwOSUETbk3vZmJ11cBVcjrvSjHHyi8aKh9KMsd1qSHPSi9lpxGenXtWcNN28KbVqc1J7qYPNSZVxHahyc9JQ7yCjK0JTNyetJ0cSSvn0lDuBj8zpvc0YkhAtrDjPbMa4Cs7uMJoxhg+HKS/DoNnSjFE0iIebG72CO9MruLqGIdeq2pw0MTeNlPz0Ks1Jsyiy55JfctmB5qTeuGlfPCy+hHp2JWbuy1cakDYTRxLKOvPO9ELgcaVUKFB9MVEIV2Erg3M7jIvrx76A0nzw7wY3/9y4LhJ6vbMjFO2MxWKhs28gnX0DGUOtvXIrZBcVcC4zhYtZV5qTJuensTd1O8XWc5SRRxlJXCrMp7i01PkJRWu9VCn1MpCjtbYppfKBGc0alRDNQWujjHUwGg6vhrxkY531IXNhyD3QdTS4QJ1aiHL+Xt4M6xLJsC6R/OfCcb65+D1xWVvRbplg96Kb5408OGgOcwfd1CLXYBy5KO8OPACMN0tdXwNvN3NcQjSdrItwKMZsxni8UjPG+cZnN9eacSNEucu5mazYtZptCRsotJ5Ga4WfdSBTui1myZhZBHRo2Wt+jpS83sK4jvIX8/ED5nOLmisoIRqtMOtKM8YL3xrPdRsL016FATObtBmjEC2pzGbj3f1biDn+GUlle1GWUqw6jNEB97Nk9HyHl1JoDo4klFFa66GVHm8zF7sSonUpK4HTW4yL6yc2Gc0Yg3vDrc8Z94w0UzNGIVrCdxeO8efvozmcvRXtlgU2LyK9xvPg4DnMGXhjq5hW7EhCsSmlemmtzwCYd8pXP7dNiJamNSR8bzZj/NRsxhgCIxfC0PkQ7vrNGEX7lZiTwYpdq9lxaQOF1jNorfC3DmRq98d4cvSMFi9p1cWRhPJLYLtS6iyggO7AT5o1KiHqkn7GKGfFRUPmOaMZY7+7jIvrvW5t980YhesqKSvjvR++Iub4Z1wu+x5lKcOqOzEm4EGeGj3f4ftgnMGRWV5blVJ9gL4YCeU4MKy5AxPiGhXNGKONUQnKuGN9/C+lGaNwed+cO8pb+1ZxOGc72poFtg706DCBBYPmMHvg2FZR0qqLQwsLaK2LgYpFtZRSH2OsRSJE8yotNBaniouB018ZzRg7DYLJ/2M0Y/QLd3aEQjRYQnYGr+3+mB2XNlBkPWuUtCyDuCvycZ4YPcPlVsBs6Eo1UpQWzaeiGeMqOLr2SjPGMY+bzRgHOTtCIRqspKyMd/Z/yeoTn5FcFmuWtDpzY+ACloyez8BOXZ0dYoM1NKG0nxbFouWkHDN6aB1afaUZ44AZRjPGyHHSjFG4tB1nD/P2vmiO5m5HW7PB1oGeHW5l4ZC5zOw/2iVKWnWpMaEopdZRfeJQQHCzRSTal9zLRgKJWwWXDxnNGHtPNJsx3gkerjXkF6Kyi1nprNgdw9eJGym2nkNrRYBlMHdFTufx0Xe7XEmrLrWNUJY3cJsQtSvOg+NfGKORc1+bzRhHwNQ/wMDZ4BPq7AiFaLCSsjL+vm8Tq0+sIcW2D2Upw0134aaghTw1er7Di4a5ohoTitb665YMRLRxtjKjk29ctJFMSgsgoBuM+3/GdZGQPs6OUIhG2X72EG/vi+ZY7g6zpOVNrw638ZOhc5ne74Y2UdKqS0OvoQhRN60h6YAxQ+vQashPAa8AI4EMmQ/dxshNh8KlXcxK59VdMexMKi9pWQi0DObuHtN5bPTd+Hp2cHaILUoSimh6WfHmTYcxkHYCrB5XmjH2uV2aMQqXVlRawjv7NvPJyTWk2PabJa1wxgU9xJIxc+kX2nZLWnWRhCKaRkUzxmhjyi9Atxth2gpjppY0YxQubtuZOKOklbcDrDlg86a39yQeHjqXu/pGtYuSVl0caV9f3WyvbCAW+KvWWhbbaq/KSoybDQ+ugpObzWaMfeC254ybDgMjnR2hEI0Sn5XKioqS1gWjpGUdwvSe03n0hmntrqRVF0dGKGeBUOAj8/F8IBm4HvgbRjt70V5oDRf3GiORI59CYabRjDHqJ0ZJK3y4XBcRLq2otIS/xW7k01NrSLX9gLLYKkpaT42ZT99Q6c5QE0cSynCt9fhKj9cppXZqrccrpY405E2VUkFANBAJnAfmaa0zq+wzDGPdFT+M7sYvaa2jzW0TgT8CFiAPWKi1Pt2QWISD0s8YSSQuGjLPg1sHoxnj0Hug5wRpxihc3lenDrDyhxiO5+0Aa65Z0prMw0PncVffkVLScoAjCSVUKdVNax0PoJTqBoSY20oa+L5Lga1a62VKqaXm42er7FMAPKi1PqWUCgf2KaU2a62zMBLNDK31MaXU48BzwMIGxiJqkp9mtISPi4ZLsYCCnrfALc8azRg9fZ0doRCNcj4jhVd3x/Dt5U2UmCWtIOtQZvSawaOjptHRUyaQ1IcjCeX/Ad8qpc5g3CXfA3hcKdUReL+B7zsDmGB+/T6wgyoJRWt9stLXiUqpFIzSWxbGNZ3y1rL+QGID4xBVlRbCiQ1mM8YtZjPGwTD5d8YiVdKMUbi4gtJi/v79Jj49tYY0/QNK2XAngvHBD/P0mPn0Ceni7BBdliPt6zeY7ev7Ybavr3QhfkUD37eT1jrJPH6SUiqstp2VUjcAHsAZ86lFwAalVCGQA4yp5bWLgcUA3bpJg+Rq2e3GMrkHo42ZWiW54BsOY58wrot0GujsCIVotM2nfmDl/mhO5u80S1odud77Dh4ebszSEo3nyCwvd+ARoPw6yg6l1F+11qV1vG4L0LmaTb+pT4BKqS7Ah8ACrbXdfPoZ4E6t9R6l1C+BV6hhjXut9UpgJUBUVJQ0taws+ahRzjr0MeRcAg9fGDDdSCKRN0szRuHyzmYks2JXDN8lb6TEetEsaQ1jZq8ZPDLqLilpNTFHSl5vAe7AX8zHD5jPVfsLvJzWelJN25RSyUqpLubopAuQUsN+fsB64Dmt9W7zuVBgqNZ6j7lbNLDJgfMQADlJcHi1MRpJLm/GOAlu/x1cP1WaMQqXV1BazMrvN7Lm1BrS9AGzpNWVCSE/5ekx8+gVXN3fuaIpOJJQRmmth1Z6vE0pdbCR77sWWAAsMz9/XnUHpZT/dBt1AAAgAElEQVQH8Bnwgdb640qbMgF/pdT15nWWycCxRsbTthXnwbF1xmikvBnjdSOlGaNoUzad3M/ffojhZP7XYM0Dmw99O07h4WFzubPvSGeH1y44klBsSqleWuszAEqpnhjTeBtjGRCjlHoYiAfmmseOAh7VWi8C5mGU2YKVUgvN1y3UWh9QSv0U+EQpZcdIMA81Mp62x1YGZ7ebzRjXm80Yu8O4X5jNGHs7O0IhGu1sRjKv7ormu+RNlFovorWVEOswZvaZyeJRU/F2l5JWS1Ja135Zwbzn412MGxwV0B34idZ6e/OH17SioqJ0bGyss8NoPlpD4g/GDK3DqyE/1WjGOGi2kUS6jpabDoXLKygt5u0961lz5nMy9EGUsuFh68pNnaby9Nh59Azq5OwQ2xyl1D6tdZ0zFxyZ5bXVnOXVF3OWFzCs8SGKJpN5AQ6VN2M8aTZjnGI2Y5wszRhFm7DhxD7+/kMMpwp2VpS0+vlM5afD53FHn+HODk/gYHNIrXUxEFf+WCn1MSBzcJ2pMNOY4nswGuL/YzzX7Ua4+wmjGWOHQOfGJ0QTOJWWxIrd0exK2UypNcEsaQ1ndp+ZLBo1RUparUxDuw1L3cQZyorh1FfGcrknN4OtBEKuh9v+22zG2N3ZEQrRaPnFxbz9/Rd8XlHSsuNBdyaGPcrTo+cSGVTrbWvCiRqaUOR+jpaiNVzcY1xcP/wpFGVBx1CIehiGzocuw+S6iHB5drud9Sf28c7BGM4UfAPWfLD50t/nLhYPn8fkPlJldwU1JpQa2taDMToJbraIhCHt9JVmjFkXjGaM/acZ10V63gpWWcpGuL5TaUm8umsVu1I2U+Z2CW23Euo2gh9dP4tFI+/Ay93D2SGKeqjtt9LyBm4TDZWfBoc/MZsx7gNlgR63wIRfG8lEmjGKNiC/uJi/7F3L2rNrydRxKGXHU3Xn1rBHeXrsPLoFyH1RrqrGhKK1/rolA2m3SgqubsaobdB5MNz+exg0B/ykUZ1wfUZJK5Z3Dn7MmYKdYC0Amy8DfKaxeMQ8JvUeWvdBRKsndRNnsNvg/LfGSOToWqMZo991cOPPzGaMA5wdoRBN4nhqAq/tjmF3ymbK3BLRdithbiP50fUzeVhKWm2OJJSWlHzEWC730GrITTSbMc4wLq53vxlkAR/RBuQWF/LWnnWsO7uWTA6ZJa1Ibg17nKfHzpGSVhtWr4SilOqstb7cXMG0STmJRgKJi4bkw2BxM5ox3vES9J0K7rImtXB9druddce/5x8HYzhb+K1R0rL7MdD3bh4dOZ9bew52doiiBdR3hLIBGNEcgbQpxblXmjGe/RrQcF0UTP2j0QalY0idhxDCFRxPTWDF7mj2pGymzC0JbXcjzG0kc/rO5KERt0tJq52pb0KRGx5qYiuDM9uuNGMsK4TASLjlVzB4njRjFG1GbnEhf96zli/OriWLw2ZJqwcTOz3BU2Pm0jVA7ipor+qbUP7WLFG4qopmjNHGdN/8VKPlybD7zGaMN8hNh6JNsNvtrDm2h/fiPuZc4bdgLUTZ/RnkO51HR85nQs9Bzg5RtAL1Siha67/UvVc7UN6M8WA0pJ+60oxx6D3QezK4yTBftA3HUhJYsXsVe1M3U+Z2GW13o5N7FPP6zuInI27Hw03m9Ygr5KfBUYWZcGSNMRqJ32U81/0mY6rvgBnQIcC58QnRRLKLCvjzns9Zf24d2RxGKY2X6smkzk/y1Ji5RPgHOTtE0UpJQnHEFz+HHz40mzH2hYm/NZoxBkjDZdE22O12Pju2m/fiPuZ84XdmSSuAIX4zeXTkfMb3GOjsEIULkITiCL9wGLXIuC7SZahcFxFtxuHL8by2J5rv0zZjc0tG293o7D6Kef1msXD4ZClpiXqRnxZHjP+FsyMQoslkFxXw5u41bDi/jmyOoJSmg+rFrZ3n89TYHxHuJyUt0TCSUIRoB+x2O6uP/IcPDq3mQtF3YC0yS1qzeWLUfG7q3t/ZIYo2QBKKEG3Y4cvxrNjzEbFpX2JzS0Hb3eniPop5/WazYPhEKWmJJiU/TUK0MVmF+by++zM2nf+CHHXULGn15rYu9/D02Dl09pXloUXzkIQiRBtgt9v5+PB3fHB4NfHF/wFLEcoeyNCA2TwRdQ83du/n7BBFOyAJRQgXFnf5PK/tjmZf+pWSVrjbDdzTfzYPDp+Im9Xq7BBFOyIJRQgXk1mQxxt71rDp/Dpy1DGU0nhb+jCxy30sGTtbSlrCaZyWUJRSQUA0EAmcB+ZprTOr7NMd+BSwAu7AG1rrt81tI4H3gA4YXZCf0lrrFgpfiBZlt9uJPvQN/zzyCfHFu8ySVhDDA+bwxKj5jOnW19khCuHUEcpSYKvWeplSaqn5+Nkq+yQBN2qti5VSPsBhpdRarXUi8BawGNiNkVCmABtbLnwhmt8Pied4Y+8q9qV/hd0t1ShpuY/m3v6zeWDYbVLSEq2KMxPKDGCC+fX7wA6qJBStdUmlh56ABUAp1QXw01rvMh9/AMxEEopoAzIL8nht96dsvvAFueq4WdK6nknh97Nk7I/o5OPv7BCFqJYzE0onrXUSgNY6SSkVVt1OSqmuwHqgN/BLrXWiUioKSKi0WwJwXXMHLERzsdvtfHRoJ/88/AkXS3ahLMVmSWsuT466h9Hd+jg7RCHq1KwJRSm1BehczabfOHoMrfVFYIhSKhxYo5RaTfULfVV7/UQptRijNEa3btLMUbQu+y6d4c290ezP+Aq7Wxra7sF17qO5b8CP+PHQCVLSEi6lWROK1npSTduUUslKqS7m6KQLkFLHsRKVUkeAccB3QESlzRFAYg2vWwmsBIiKipKL9sLp0gtyeX3Xp2yO/4J8y3EAOlr6Mum6B3lqzI8I9fFzcoRCNIwzS15rgQXAMvPz51V3UEpFAOla60KlVCBwE/CKmYRylVJjgD3Ag8AbLRe6EPVjt9v518Ed/Pvop1ws2Y2yFGOxBzPC/x5+dsN8oiJkiWjh+pyZUJYBMUqph4F4YC6AeX3kUa31IqA/8CellMYocy3XWh8yX/8YV6YNb0QuyItWKDbhNG/sjeZA5lfY3dLRdk8i3Efz44E/4t4ht0hJS7Qpqj3duhEVFaVjY2OdHYZo41Lzcnht9ydsubiefMsJADra+3FHt2ksGTubYG9fJ0coRP0opfZpraPq2k/ulBeiCZTZbGZJ6xMule5BWUqw2IMZGXAPT46SkpZoHyShCNEI3yec5o29qziYuaWipNXV40Z+PHA29w25BYvF4uwQhWgxklCEqKfUvBxW7FrNloT1FFhOorXC19KPO7o+xM/GzJKSlmi3JKEI4YAym41/HtjOR8c+rVTSCmVU4H387IZ7GB7ew9khCuF0klCEqMWe+FO8+f1HHMzainbLQNs96eZxIw8MmsP8weOkpCVEJZJQhKgiOS+b13atZmvCegosp9Ba4Wfpzx1dF7FkzCwCvX2cHaIQrZIkFCEwSlof/LCVVcc+JbFsL8pSisUeyg2BP+Zno+9hWJdIZ4coRKsnCUW0a7vjT/Dm96uIy9qKdssEuxfdPcfxwKAfMW/QzU4vaZWWlpKQkEBRUZFT4xDtg5eXFxEREbi7uzfo9ZJQRLtzOTeTFbtWsy1hI4XW8pLWAKZ0W8ySMbMI6NDR2SFWSEhIwNfXl8jISJSqrieqEE1Da016ejoJCQn06NGwSSaSUES7UGaz8f4PW1h17DOSzJKWVYcxOuB+loyez5DOkc4OsVpFRUWSTESLUEoRHBxMampqg48hCUW0ad9dOMafv4/mcPZWtFsW2LyI9BrPg4PnMGfgjU4vaTlCkoloKY39WZOEItqcxJwMXtv1CdsvrafQegatFf7WgUzt/hhPjp7RqkpaQrQlrf/PMyEcUFJWxsrvNzLpw8Xc/skkNlx+nRKdx5iAB1k15Qu+W/gRz024T5JJPWVlZfGXv/ylQa9dsWIFBQUFjY4hNjaWJUuWNPo45RYuXMjq1auveT4xMZE5c+YAsGPHDqZNm1bt6yMjI0lLS2uyeMotWrSIo0eP1rpPTbGfP3+ef//73zW+btOmTfTt25fevXuzbNmyRsdaExmhCJf2zbmjvLVvFYdztqOtV0paCwfPZfbAsS5R0mrNyhPK448/Xu/Xrlixgvvvvx9vb+9GxRAVFUVUVJ2NbhstPDy82l/WLeXvf/97g19bnlDuu+++a7bZbDaeeOIJvvrqKyIiIhg1ahTTp09nwIABjQm3WvK/TbicxJwMfrV5JaP+MZPHd84nLm8Nvpau3Bv5X3x739d8cd8K5gy+SZJJE1i6dClnzpxh2LBh/PKXv+SPf/wjo0aNYsiQITz//PMA5Ofnc9dddzF06FAGDRpEdHQ0r7/+OomJidx6663ceuutNR7fx8eHZ599lpEjRzJp0iT27t3LhAkT6NmzJ2vXrgWuHi288MILPPTQQxX7vP7667XG/8EHHzBkyBCGDh3KAw88UPH8zp07ufHGG+nZs2dFEjl//jyDBg265hjp6encfvvtDB8+nEceeYTalvz4wx/+UBHTM888w2233QbA1q1buf/++wH48ssvGTt2LCNGjGDu3Lnk5eUBMGHCBMqX13jnnXe4/vrrmTBhAj/96U958skna4196dKlfPPNNwwbNoxXX331qpj27t1L79696dmzJx4eHtxzzz18/vk16xk2CRmhCJdQUlbGu/u/JObEZySXxaIsZVh1Z8YGPshTo+9hYKeuzg6x2b247ghHE3Oa9JgDwv14/u6BNW5ftmwZhw8f5sCBA3z55ZesXr2avXv3orVm+vTp7Ny5k9TUVMLDw1m/fj0A2dnZ+Pv788orr7B9+3ZCQkJqPH5+fj4TJkzg5ZdfZtasWTz33HN89dVXHD16lAULFjB9+vRrXnP8+HG2b99Obm4uffv25bHHHqv2vokjR47w0ksv8d133xESEkJGRkbFtqSkJL799luOHz/O9OnTK0pd1XnxxRe5+eab+e1vf8v69etZuXJljfuOHz+eP/3pTyxZsoTY2FiKi4spLS3l22+/Zdy4caSlpfH73/+eLVu20LFjR15++WVeeeUVfvvb31YcIzExkd/97nfs378fX19fbrvtNoYOHVpr7MuWLWP58uV88cUXFcdYtGgRGzZs4NKlS3TteuX/R0REBHv27KnxHBpDEopo1XaeO8JbsdEcyd2GtmaDrQM9O9zKwiFzmdl/tIxCWtCXX37Jl19+yfDhwwHIy8vj1KlTjBs3jl/84hc8++yzTJs2jXHjxjl8TA8PD6ZMmQLA4MGD8fT0xN3dncGDB3P+/PlqX3PXXXfh6emJp6cnYWFhJCcnExERcc1+27ZtY86cORUJLSgoqGLbzJkzsVgsDBgwgOTk5Fpj3LlzJ59++mnFewcGBta478iRI9m3bx+5ubl4enoyYsQIYmNj+eabb3j99dfZvXs3R48e5aabbgKgpKSEsWPHXnWMvXv3csstt1TEO3fuXE6ePFmv2MPDw9mwYQNAtSOq5po5KAlFtDoJ2Rms2BXDjsQNFFvPobUiwDKIuyJn8Pjou/H3alxN3lXVNpJoCVprfv3rX/PII49cs23fvn1s2LCBX//619x+++1X/cVdG3d394pfbhaLBU9Pz4qvy8rKqn1N+T4AVqu1xv201jX+4qx8DEdWrXX0F7C7uzuRkZG8++673HjjjQwZMoTt27dz5swZ+vfvz5kzZ5g8eTIfffRRjceoK576xh4REcHFixcrHickJBAeHu7A2dSf/HknWoWSsjLe2ruBiR/+lCmfTmRz8p+x6UJuClrIx3du4NuF/+bXt8xvt8nEWXx9fcnNzQXgjjvu4B//+EdFzf/SpUukpKSQmJiIt7c3999/P7/4xS/Yv3//Na91hokTJxITE0N6ejrAVSWv+hg/fjz/+te/ANi4cSOZmZl17r98+XLGjx/PuHHjePvttxk2bBhKKcaMGcN3333H6dOnASgoKLhq9AFwww038PXXX5OZmUlZWRmffPJJnTHW9r0eNWoUp06d4ty5c5SUlLBq1apqS4lNQUYowqm2nz3E2/uiOZa7o6Kk1avDbfxk6Fym97tBSlpOFhwczE033cSgQYOYOnUq9913X0WJxsfHh3/+85+cPn2aX/7yl1gsFtzd3XnrrbcAWLx4MVOnTqVLly5s3769xWMfOHAgv/nNb7jllluwWq0MHz6c9957r97Hef7557n33nsZMWIEt9xyC926dat1/3HjxvHSSy8xduxYOnbsiJeXV0UZMDQ0lPfee497772X4uJiAH7/+99z/fXXV7z+uuuu47/+678YPXo04eHhDBgwAH9//1rfc8iQIbi5uTF06FAWLlzI/PnzK66huLm58eabb3LHHXdgs9l46KGHGDiweUa7ypEhU1sRFRWly2dRCOe5mJXOit0xfJ240SxpWQhkMHf3nM5jo+/G17ODs0NsNY4dO0b//v2dHYZoYXl5efj4+FBWVsasWbN46KGHmDVrVou8d3U/c0qpfVrrOuduywhFtIii0hLe2beZT06uIcW2H2Upw02Hc3PQT3hqzDz6hV57UVWI9uqFF15gy5YtFBUVcfvttzNz5kxnh+QQSSiiWW07E8fb+42SFtYcsHnT23sSDw+dy119o6Sk1U6MHj26osRT7sMPP2Tw4MGNPnZ6ejoTJ0685vmtW7cSHBzc6OM74z2XL1/e6GM4gyQU0eTis1JZsSuGnUkbKbZeMEpa1iFM7zmdR2+YJiWtdqi57nsA4zrPgQMHmu34reU9XYEkFNEkikpL+Pu+zXxy8jNSbftRFhtuOpxxQQ/x1Jj59A1tnmmKQojWwykJRSkVBEQDkcB5YJ7WOrPKPt2BTwEr4A68obV+WynlDXwM9AJswDqt9dKWi15UtuX0Qf66P5rjeTvAmmuWtCbz8NB53NV3pJS0hGhHnDVCWQps1VovU0otNR8/W2WfJOBGrXWxUsoHOKyUWgtkAcu11tuVUh7AVqXUVK31xhY9g3YsPiuVV3bF8E3SRkrMklaQdSgzes3g0VHT6FjpxishRPvhrIQyA5hgfv0+sIMqCUVrXVLpoSfmTZha6wJge/k+Sqn9gEwRamZFpSX8LXYjn55aQ6rtB7OkdR3jgx/m6THz6RPSxdkhCiGczFn1iE5a6yQA83NYdTsppboqpeKAi8DLWuvEKtsDgLuBrc0cb7v11akDzIn5NaM+nMDKk8+RVnaMPh1uZ9mYd/nh4U38edrTkkzaMFkP5VquuB7KQw89RFhYWLXdlJtSs41QlFJbgM7VbPqNo8fQWl8EhiilwoE1SqnVWutk8/huwEfA61rrs7XEsRhYDNR5h6swnM9I4dXdMXx7eSMl1nizpDWMmb1m8Miou6Sk1Y7Ieigtp7nWQwEjET355JM8+OCDDX4PRzTbCEVrPUlrPaiaj8+BZKVUFwDzc0odx0oEjgCV25iuBE5prVfU8dqVWusorXVUaGho406qDSsoLWbFf9Yw4f2FTFt7O9tS/4rWdm4JXsTnd29m54L3+fnNsyWZtDOyHorrr4cCRn+xyt2Wm4uzrqGsBRYAy8zP16z2opSKANK11oVKqUDgJuAVc9vvAX9gUYtF3EZtOrmfv/0Qw8n8r8GaBzYf+nacwsPD5nJn35HODk9UtnEpXD7UtMfsPBim1rwkrKyH4vrrobQkZyWUZUCMUuphIB6YC6CUigIe1VovAvoDf1JKaUBhzOw6ZCaa3wDHgf1mW+k3tdYNHy+2M2czknl1VzTfJW+i1HoRra0EW4cys/dMHrnhTrzdZRQiriXrobjmeigtySkJRWudDlzTt0BrHYs56tBafwUMqWafBIwEI+qhoLSYv+7dwJrTa0jXB1HKhgdduTVkMU+PnUfPoE7ODlHUpZaRREuQ9VDq1hrXQ2lJctdZG7fhxD5mr3qW0R9O4B+nf0uG7SR9O0zhj2PfZ99DG3j9rp9JMhE1kvVQXH89lJYkrVfaoDPpl3l1VzT/Sd5EqVsCWlsJsQ5ndp+ZLBo1RUpawmGyHorrr4cCcO+997Jjxw7S0tKIiIjgxRdf5OGHH67396Iush5KG5FfXMzb33/B52c+J0MfRCk7HrZu3Nx5Ks+MmUdkULW3+ohWTtZDaZ9kPRThFOtPxPL3H2I4XbATrPlg86W/z10sHj6PyX2GOTs8IUQDyHooosWcSkvi1V2r2JWymTK3S2i7lVA3o6T106ipeLl7ODtEIa4i66HUj6yHIppVfnExf9m7jrVnPydTx6GUHU/VnVvDHuXpsfPoFiA3bYrWS9ZDaR8kobRidrud9Sdieefgx5wp2AnWArD5MsBnGotHzGNS76F1H0QIIVqIJJRW6ERqIq/tjmZXyibK3BLNktYIfnT9LBaNvENKWkKIVkkSSiuRW1zIW3vWse7cukolrUhuDXucp8fOkZKWEKLVk4TiRHa7nXXHv+cfB2M4W/itWdLyY4DvNB4dMZ/bel3TKEAIIVotuVPeCY6nJvDYulcY+e5Unvt+EWeKthHqNpjH+v0f3z+wnZi5L0kyEa2CrIdyLVdcDyUrK4s5c+bQr18/+vfvz65duxodb3VkhNJCcosL+fOetXxxdi1ZHDZLWj24rdPjPD1mHl0Dmmd6oxCNIeuhtJzmXA/lqaeeYsqUKaxevZqSkpImSfTVkYTSjOx2O2uO7eH9uNWcLfwGrIUouz+DfKfz6Mj5TOjZvKunibbl5b0vczzjeJMes19QP5694dkat1deD2Xy5MmEhYURExNDcXExs2bN4sUXXyQ/P5958+aRkJCAzWbjv//7v0lOTq5YDyUkJKTG1is+Pj488cQTbNmyhcDAQP73f/+XX/3qV8THx7NixQqmT5/Ojh07Klqzv/DCC8THx3P27Fni4+N5+umnax29fPDBByxfvhylFEOGDOHDDz8EjA7Cr7zyCpcvX+YPf/gDc+bM4fz580ybNo3Dhw9fdYz09HTuvfdeUlNTueGGG+pcD8XLy4slS5bwzDPPcPDgQbZt28bWrVt59913+ec//8mXX37J888/T3FxMb169eLdd9/Fx8eHCRMmsHz5cqKionjnnXd4+eWXCQ8Pp0+fPnh6evLmm2/WGPvSpUs5duwYw4YNY8GCBTzzzDMVMeXk5LBz586KtjMeHh54eDTPxB4peTWDYykJPLJ2OSPfvYPnYxdzpmgbYe5Debz/y8Q+uINVc38nyUS4hGXLltGrVy8OHDjA5MmTOXXqFHv37uXAgQPs27ePnTt3smnTJsLDwzl48CCHDx9mypQpLFmyhPDwcLZv315rH6/y9VD27duHr69vxXoon332WY0di48fP87mzZvZu3cvL774IqWlpdXuV74eyrZt2zh48CCvvfZaxbbyNUW++OILli5dWuv3oHw9lB9++IHp06cTHx9f477jx4/nm2++AYxSXV5eXo3roezfv5+oqCheeeWVq45Rvh7K7t27+eqrrzh+/Oo/IqqLfdmyZYwbN44DBw7wzDPPkJiYyJ133gnA2bNnCQ0N5Sc/+QnDhw9n0aJF5Ofn13rODSUjlCaSXVTAn/d8zoZz68ySlsZL9WRS5yd5asxcIvybf7U00bbVNpJoCbIeimuuh1JWVsb+/ft54403GD16NE899RTLli3jd7/7Xa3n3RCSUBrBbrfz2bHdvBf3MecLvzNLWgEM9p3JY1HzGd9joLNDFKLJyHoodWuN66FEREQQERHB6NGjASpWeGwOUvJqgCPJF1m89o+MePcOXoh9hHNFO+jkPoyfDfgDsQ9u56O5/yPJRLQJsh6K66+H0rlzZ7p27cqJEycAo9/YgAED6jxmQ8gIxUHZRQW8uXsNG86vI5sjKKXpoHoxofPPeHrsHML9pKQl2h5ZD6VtrIfyxhtv8OMf/5iSkhJ69uzJu+++W+/vgyNkPRQH3Pvxbzmcu8koadkCGOQ3kcdGzmNcj+bJ8kKUk/VQ2idZD6UNs2kbnd2HM6/fbBYMn4iHm3zbhBDNR9ZDacNi5r3k7BCEcGmyHkr9yHooQghRA1kPpX2QWV5CtHLt6TqncK7G/qxJQhGiFfPy8iI9PV2Simh2WmvS09Px8vJq8DGk5CVEKxYREUFCQgKpqanODkW0A15eXtV2HXCU0xKKUioIiAYigfPAPK11ZpV9ugOfAlbAHXhDa/12lX3WAj211tIcS7Q57u7u9OjRw9lhCOEQZ5a8lgJbtdZ9gK3m46qSgBu11sOA0cBSpVR4+Ual1GwgryWCFUIIUTtnJpQZwPvm1+8D10y01lqXaK3L5xp6UilepZQP8HPg980cpxBCCAc4M6F00lonAZifw6rbSSnVVSkVB1wEXtZaJ5qbfgf8Cah1pRil1GKlVKxSKlbq0EII0Xya9RqKUmoL0LmaTb9x9Bha64vAELPUtUYptRroAvTWWj+jlIqs4/UrgZVmPKlKqQuOvncVIUDTr/vZusk5tw9yzm1fY8+3uyM7NWtC0VpPqmmbUipZKdVFa52klOoCpNRxrESl1BFgHBAKjFRKncc4hzCl1A6t9YQ6jhFa33OoFG+sI71s2hI55/ZBzrnta6nzdWbJay2wwPx6AfB51R2UUhFKqQ7m14HATcAJrfVbWutwrXUkcDNwsq5kIoQQonk5M6EsAyYrpU4Bk83HKKWilFJ/N/fpD+xRSh0EvgaWa60POSVaIYQQtXLafSha63Tgmu5qWutYYJH59VfAkDqOcx5oiXtQVrbAe7Q2cs7tg5xz29ci59uu1kMRQgjRfKSXlxBCiCYhCaUKpdQzSqkjSqnDSqmPlFJeVbZ7KqWilVKnlVJ76pq27AocOOefK6WOKqXilFJbzZY4Lq2uc6603xyllFZKufSMIEfOVyk1z/x3PqKU+rcz4mxKDvxcd1NKbVdK/WD+bN/prFibilLqKfN8jyilnq5mu1JKvW7+/opTSo1o0qAq46gAAAYcSURBVAC01vJhfgDXAeeADubjGGBhlX0eB942v74HiHZ23C1wzrcC3ubXj7WHczaf9wV2AruBKGfH3cz/xn2AH4BA83GYs+NugXNeCTxmfj0AOO/suBt5zoOAw4A3xvXxLUCfKvvcCWwEFDAG2NOUMcgI5VpuQAellBvGP0xile2VW8asBiYqpVQLxtccaj1nrfV2rXV5R4LdQMPbkbYedf07g9GN4Q9AUUsG1kzqOt+fAn/WZoNWrXWt94W5iLrOWQN+5tf+1Wx3Nf2B3VrrAq11GcbM2KoL0c8APtCG3UCAeR9gk5CEUonW+hKwHIjHaEyZrbX+sspu12G0gcH8R8sGmmed0Rbg4DlX9jDGXzguy5FzVkoNB7pqrb9wQohNysF/4+uB65VS3ymldiulprR0nE3JwXN+AbhfKZUAbAB+1qJBNr3DwHilVLBSyhtjNNK1yj4Vv79MCeZzTUISSiXmzZMzgB5AONBRKXV/1d2qeanLTpVz8JzL970fiAL+2HIRNr26zlkpZQFeBf6fcyJsWg7+G7thlL0mAPcCf1dKBbRknE3JwXO+F3hPax2B8cv3Q/Pf3iVprY8BLwNfAZuAg0BZld2a9feXy37zmskk4JzWOlVrXYqxFsuNVfZJwMz65lDaH8ho0SibliPnjFJqEkYPtun6SgdoV1XXOfti1KN3mO19xgBrXfjCvKM/159rrUu11ueAExgJxlU5cs4PY1xbQWu9C/DC6HnlsrTW72itR2itx2P8XjpVZZeK31+mCJqw1CcJ5WrxwBillLd5XWQicKzKPpVbxswBtmnzapeLqvOczfLPXzGSSVuordd6zlrrbK11iNY6UhvtfXZjnHusc8JtNEd+rtdgTL5AKRWCUQI726JRNi1HzjnefB6lVH+MhOLSLcmVUmHm527AbOCjKrusBR40Z3uNwSgFJjXV+0tCqURrvQfjQvt+4BDG92elUup//n979xMaVxXFcfz7s3WRGgga3Lhos2hoQJRUwU1LFrUU3BQCAcF0ERdiIIigdqGIQQu1tBR10za0VQMGkeqmpYKBEJEkCzVBEixBC7px19KFKV0lp4t7pwwhGcbk5s/U3wcCL/fde999Mwxn3puZcyQdzd0uA62SbpLqsaxUGKxh1HnOZ4Bm4Iqk35SqZDasOs/5oVHn+f4A3JZ0AxgHjkfKZtGQ6jznt4HXlFI7fU36FlgjvzkE+C4/h9eAgYi4I6lfUn/e/z3pjcJN4CLpW6vF+JfyZmZWhK9QzMysCAcUMzMrwgHFzMyKcEAxM7MiHFDMzKwIBxQzMyvCAcVsm5H0d/5x4VrG9kl6qsRcZv+VA4rZw6WPlLvKbNM5oJitQlKbpHlJl3LRohFJh3NG3j8lvZD/pnKRpilJ+/LYtyR9nrefyeN3rXKcVkmjeY4hqhL4STom6eecoWBI0o7cviDprKQZpaJnT0rqISXvHMn9m/I0b+R+c5I6NvIxs/83BxSz2vYCnwHPAh3AK8BB4B3gPWAe6IqI/cAHwMk87lNgr6Ru4Avg9aqaMssNAhN5jqvAbniQX+pl4EBEdAKLQG8e8xgwExHPkepeDEbEt8CvQG9EdEbEvdz3Vu53Pq/bbEPs3OoFmG1zf0XEHICk34GxiAhJc0AbKdv0sKR2UhrwRwEiYklSHzALDEXEZI1jdJES+RER1yXdye0vAs8Dv+Qabk1AJTnnEvBN3v6KlE13NZV905XjmG0EBxSz2qpT9S9V/b9Eev2cAMYjoltSG/BjVf92YIH6PtNYKamegOGIeHeN4ysqa17Er3nbQL7lZbY+LcA/ebuv0iiphXSrrIuUnbqnxhw/kW9lSXoJeDy3jwE9VSnJn5C0J+97hFQ+AdJtuIm8/S+pnovZpnNAMVuf08DHkiaBHVXtnwDnIuIPUiGnU5XAsIIPSaVbZ4AjpDodRMQN4H1gVNIsqRJfpf73XeBpSdPAIeCj3P4lcGHZh/Jmm8Lp680akKSFiGje6nWYVfMVipmZFeErFLNNIulV4M1lzZMRMbAV6zErzQHFzMyK8C0vMzMrwgHFzMyKcEAxM7MiHFDMzKwIBxQzMyviPsDzENRqN6JKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2418026c128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch2_2.best_score_, gsearch2_2.best_params_))\n",
    "test_means = gsearch2_2.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch2_2.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch2_2.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch2_2.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch2_2.cv_results_).to_csv('my_preds_maxdepth_min_child_weights_2.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(min_child_weight), len(max_depth))\n",
    "train_scores = np.array(train_means).reshape(len(min_child_weight), len(max_depth))\n",
    "\n",
    "for i, value in enumerate(min_child_weight):\n",
    "    pyplot.plot(max_depth, test_scores[i], label= 'test_min_child_weight:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'max_depth' )                                                                                                      \n",
    "pyplot.ylabel( '- Log Loss' )\n",
    "pyplot.savefig( 'max_depth_vs_min_child_weght2.png' )"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "对max_depth和min_child_weight进行细调之后得到max_depth的最佳参数为9,min_child_weight的最佳参数为0。\n",
    "并且根据测试结果描绘了对应的图示，但是此处出现折线的情况似乎为数据异常，因为程序运行时间较长，这里暂时先不进行二次修改。\n",
    "之所以对这两个参数进行再一次的细调是因为这两个参数的值会严重影响模型性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "def modelfit(alg, X_train, y_train, useTrainCV=True, cv_folds=None, early_stopping_rounds=100):\n",
    "    \n",
    "    if useTrainCV:\n",
    "        xgb_param = alg.get_xgb_params()\n",
    "        xgb_param['num_class'] = 9\n",
    "        \n",
    "        xgtrain = xgb.DMatrix(X_train, label = y_train)\n",
    "        \n",
    "        cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], folds =cv_folds,\n",
    "                         metrics='mlogloss', early_stopping_rounds=early_stopping_rounds)\n",
    "        \n",
    "        n_estimators = cvresult.shape[0]\n",
    "        alg.set_params(n_estimators = n_estimators)\n",
    "        \n",
    "        print (cvresult)\n",
    "        #result = pd.DataFrame(cvresult)   #cv缺省返回结果为DataFrame\n",
    "        #result.to_csv('my_preds.csv', index_label = 'n_estimators')\n",
    "        cvresult.to_csv('my_preds4_2_3_699.csv', index_label = 'n_estimators')\n",
    "        \n",
    "        # plot\n",
    "        test_means = cvresult['test-mlogloss-mean']\n",
    "        test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "        train_means = cvresult['train-mlogloss-mean']\n",
    "        train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "        x_axis = range(0, n_estimators)\n",
    "        pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "        pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "        pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "        pyplot.xlabel( 'n_estimators' )\n",
    "        pyplot.ylabel( 'Log Loss' )\n",
    "        pyplot.savefig( 'n_estimators4_2_3_699.png' )\n",
    "    \n",
    "    #Fit the algorithm on the data\n",
    "    alg.fit(X_train, y_train, eval_metric='mlogloss')\n",
    "        \n",
    "    #Predict training set:\n",
    "    train_predprob = alg.predict_proba(X_train)\n",
    "    logloss = log_loss(y_train, train_predprob)\n",
    "\n",
    "        \n",
    "    #Print model report:\n",
    "    print (\"logloss of train :\" )\n",
    "    print (logloss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     test-mlogloss-mean  test-mlogloss-std  train-mlogloss-mean  \\\n",
      "0              1.945495           0.000882             1.941117   \n",
      "1              1.766003           0.001070             1.758182   \n",
      "2              1.624965           0.000616             1.614069   \n",
      "3              1.509089           0.000075             1.495417   \n",
      "4              1.411077           0.000727             1.394634   \n",
      "5              1.327397           0.000313             1.308604   \n",
      "6              1.253882           0.000135             1.232749   \n",
      "7              1.189551           0.000292             1.165968   \n",
      "8              1.133204           0.000734             1.107498   \n",
      "9              1.083104           0.001544             1.055546   \n",
      "10             1.037433           0.001899             1.007835   \n",
      "11             0.996933           0.001892             0.965621   \n",
      "12             0.960332           0.001922             0.927275   \n",
      "13             0.927180           0.001850             0.892361   \n",
      "14             0.896956           0.002038             0.860401   \n",
      "15             0.869383           0.001988             0.831277   \n",
      "16             0.844412           0.001903             0.804758   \n",
      "17             0.821798           0.001866             0.780548   \n",
      "18             0.800962           0.001944             0.757946   \n",
      "19             0.782164           0.001943             0.737792   \n",
      "20             0.764789           0.002101             0.719091   \n",
      "21             0.748585           0.002153             0.701381   \n",
      "22             0.733341           0.002059             0.684603   \n",
      "23             0.719587           0.002299             0.669559   \n",
      "24             0.707167           0.002144             0.656071   \n",
      "25             0.695418           0.002122             0.642973   \n",
      "26             0.684707           0.002419             0.630995   \n",
      "27             0.674691           0.002452             0.619827   \n",
      "28             0.664948           0.002385             0.608765   \n",
      "29             0.656256           0.002629             0.599188   \n",
      "..                  ...                ...                  ...   \n",
      "970            0.299385           0.002199             0.040692   \n",
      "971            0.299363           0.002191             0.040624   \n",
      "972            0.299348           0.002204             0.040563   \n",
      "973            0.299361           0.002159             0.040505   \n",
      "974            0.299364           0.002131             0.040451   \n",
      "975            0.299408           0.002120             0.040393   \n",
      "976            0.299404           0.002146             0.040330   \n",
      "977            0.299372           0.002148             0.040261   \n",
      "978            0.299395           0.002178             0.040187   \n",
      "979            0.299393           0.002158             0.040121   \n",
      "980            0.299382           0.002171             0.040076   \n",
      "981            0.299376           0.002233             0.040015   \n",
      "982            0.299361           0.002256             0.039927   \n",
      "983            0.299313           0.002232             0.039840   \n",
      "984            0.299339           0.002218             0.039780   \n",
      "985            0.299308           0.002256             0.039710   \n",
      "986            0.299275           0.002237             0.039634   \n",
      "987            0.299265           0.002238             0.039575   \n",
      "988            0.299270           0.002301             0.039520   \n",
      "989            0.299340           0.002294             0.039459   \n",
      "990            0.299323           0.002264             0.039388   \n",
      "991            0.299351           0.002264             0.039313   \n",
      "992            0.299335           0.002239             0.039240   \n",
      "993            0.299349           0.002247             0.039183   \n",
      "994            0.299338           0.002264             0.039112   \n",
      "995            0.299311           0.002283             0.039053   \n",
      "996            0.299297           0.002269             0.038985   \n",
      "997            0.299309           0.002194             0.038919   \n",
      "998            0.299361           0.002203             0.038863   \n",
      "999            0.299389           0.002193             0.038809   \n",
      "\n",
      "     train-mlogloss-std  \n",
      "0              0.001777  \n",
      "1              0.001797  \n",
      "2              0.001397  \n",
      "3              0.001272  \n",
      "4              0.001220  \n",
      "5              0.001112  \n",
      "6              0.001031  \n",
      "7              0.000824  \n",
      "8              0.000766  \n",
      "9              0.000981  \n",
      "10             0.000793  \n",
      "11             0.000786  \n",
      "12             0.000540  \n",
      "13             0.000858  \n",
      "14             0.000493  \n",
      "15             0.000813  \n",
      "16             0.000542  \n",
      "17             0.000751  \n",
      "18             0.001037  \n",
      "19             0.001290  \n",
      "20             0.001194  \n",
      "21             0.001234  \n",
      "22             0.001365  \n",
      "23             0.001178  \n",
      "24             0.001567  \n",
      "25             0.001696  \n",
      "26             0.001374  \n",
      "27             0.001306  \n",
      "28             0.001193  \n",
      "29             0.000996  \n",
      "..                  ...  \n",
      "970            0.000832  \n",
      "971            0.000839  \n",
      "972            0.000844  \n",
      "973            0.000831  \n",
      "974            0.000823  \n",
      "975            0.000827  \n",
      "976            0.000816  \n",
      "977            0.000844  \n",
      "978            0.000830  \n",
      "979            0.000839  \n",
      "980            0.000842  \n",
      "981            0.000833  \n",
      "982            0.000837  \n",
      "983            0.000850  \n",
      "984            0.000840  \n",
      "985            0.000831  \n",
      "986            0.000844  \n",
      "987            0.000854  \n",
      "988            0.000842  \n",
      "989            0.000847  \n",
      "990            0.000852  \n",
      "991            0.000854  \n",
      "992            0.000855  \n",
      "993            0.000854  \n",
      "994            0.000884  \n",
      "995            0.000879  \n",
      "996            0.000874  \n",
      "997            0.000867  \n",
      "998            0.000860  \n",
      "999            0.000861  \n",
      "\n",
      "[1000 rows x 4 columns]\n",
      "logloss of train :\n",
      "0.04325725796044186\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEXCAYAAAC+mHPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcXFWd///Xp5beO0kn6YSsJEBYFQJGFnEkbhgRwW2QKAhujI7+3MdRv46OuDvzcxsdFRVxBRdQUUHkCygisiQYtrCFJIQm+9rpdHqpqs/3j3M6Xd2p6q5Ourq6O+/n43Efde+55957bt2kPn3Ovfccc3dEREQGk6h0AUREZGxQwBARkZIoYIiISEkUMEREpCQKGCIiUhIFDBERKYkChkiJzOxjZva9SpdDpFIUMMYRM2sws7Vm9oa8tEYzW2dmr8tLW2RmvzezHWa208xWmtlnzawprr/UzLJm1han1Wb2zjKXfbGZtZTzGENRqDzu/jl3f1uZjrfWzF5Sjn2Xw0hdr7H2vYx3ChjjiLu3AZcBXzOz5pj8JWCZu/8KwMyeB/wZ+BtwrLtPApYAGeCkvN393d0b3L0BeB3wJTM7eWTORIbCzFKVLoMcItxd0zibgKuAq4HFwDZgRt66O4D/GWT7S4E7+qXdA7whb/k84GFgJyEAHZe37riYtjPmOS9v3TnASmA38AzwIaAe2AvkgLY4zSxyXt8E/hC3vxs4soTv41jgZmA78BhwwYGUB/hP4Cdxu3mAA28GngZ2AO8Angs8EM/9G3nHORK4NV6PrcBPgUlx3Y/jsfbGY324hO94LfDv8VidQCouPxPP5THgxQW+i9OBjUAyL+3VwANx/lRgGdAKbAK+XOQ7XQy0FFk3EfgRsAV4Cvg4kIjrksD/H7+DNcC74/eYKrKvtcBLiqx7O7AqXtfre/7NAAZ8BdgM7Irf0bOKXe9K/38dS1PFC6CpDBcVmoAN8T/lm/PS64EssHiQ7S8lL2DEH8GdwNFx+WhgD/BSIA18OP7HrYrLq4CPxeUXxf+cx8RtNwD/lFfOU+J80R+gvHJcFX8cTo0/kD8Frhlkm3rCD/qb4zanxO/lhKGWh8IB49tADXA20AH8BpgGzIo/WGfF/EfF76saaAZuB76at+8+P4wDfcd5+VcAc4Ba4Jh4njPzylcwmAJPAi/NW/4l8JE4/3fg4jjfAJxeZB9FrxchWPwWaIzleBx4a1z3DsIP9uz4ff9fDiBgxH9XW+P1rAb+B7g9rnsZsByYRAgexxH/aCp2vTWVNqlJahxy9x2Ev0zrgOvyVjURmiE39iSY2ZfifYw9ZvbxvLynx/Q2Qu3ix8ATcd3rgT+4+83u3g38N+FH63mEv2AbgC+4e5e73wr8Hlgat+0GjjezCe6+w93vG+LpXefu97h7hhAwFg6S/1xgrbv/wN0z8XjXEprZhqM8n3b3Dnf/E+EH/mp33+zuzwB/BU4GcPdV8fvqdPctwJeBswbY70DfcY+vu/vT7r6X8IdAdTyXtLuvdfcni+z7auL1MLNGwl/dV+d9H0eZ2VR3b3P3u4byZZhZMpb9o+6+293XEmoUF8csFwBfc/eW+O/0C0PZf543Ale6+33u3gl8FDjDzObFc2gk1CzN3R9x9w1553cw1/uQpoAxDpnZRYS/7P4v8MW8VTsITR8zehLc/cMe7mP8mvAXeI+73H2Sh3sYhwEnAJ+L62YSmhp69pEj/HU7K657Oqb1eCquA3gt4QfqKTP7i5mdMcTT25g3304ITgM5HDgtBr+dZraT8GNz2DCVZ1Pe/N4Cyw0AZjbNzK4xs2fMrBX4CTB1gP0O9B33eDpv/SrgfYRa0OZ4rJlF9v0z4DVmVg28BrjP3XuO9VZC7eZRM7vXzM4doIyFTCXULJ/KS8u//jPzy91vfij6fz9thOa+WfGPlG8Qmi83mdkVZjYhZj3Y631IU8AYZ8xsGqH99u3AvwAXmNkLANx9D6Hd/zVD2ae7byL8Vf7KmLSe8EPcc0wjNI08E9fNMbP8f1tz4zrc/V53P5/QbPMb4Bc9hxlKmYbgaeAvMfj1TA3u/s4RLs/n4z5PdPcJwEWE5pIe/Y830HdccBt3/5m7Pz9u5/T9YyE/30rCj+3LgTcQAkjPuifcfSnh+/gi8Cszqy/9NNlK+Cv+8Ly0fdef0CQ0O2/dnCHsO1//76cemELvv7Ovu/tzCH/oHA38W0wvdr2lBAoY4883gN+4+22xGv5h4Lvxr0ni8lvM7CMxuGBms4H5xXZoZlMIN0Yfjkm/AF5hZi82szTwQcKN1zsJAWkP8GEzS5vZYkKgucbMqszsjWY2MTaztBKaUiD8ZT7FzCYO0/fQ4/fA0WZ2cSxP2syea2bHjXB5Ggk3tHea2SziD1ieTcARecsDfcf7MbNjzOxF8Tp3EGo32UJ5o58B7wFeQLiH0bOfi8ysOdZodsbkovsxs5r8iVCD/QXw2fhI9+HABwg1qp7zeq+ZzTKzSYQb9YNJ9ztOKpb/zWa2MJ7z54C73X1tvL6nxe9tT/w+soNcbylFpW+iaBq+CXgV4S+vSf3SbwE+m7d8GnAD4QdhJ/AQ8FlgSlx/KeE/Us8TQpsJbdzT8vbxasLNy13AX4g3keO6E2Larpjn1TG9CvgjoWmsFbgXeH7edlcSmhV2Uvwpqc/kLS9mkBvlMd8xhCertsT930q49zGk8lD4pncqL38LeQ8UEH4kP573nSyP3+cKQgBoyct7PrAuHutDJXzHa+l7k/xEwr2m3YQHA35f6DvMyz+X8OP+h37pP4nXu43wB8Krimy/OJ5//+kowr2yn8Tv+2ngE/Q+JZUi1IC3EZ6Sej+hRmJFjrO2wDE+E9e9g3ADv+d8Z8f0FxOejGqj94m0hsGut6bBJ4tfsIjIiDOzlwPfdvfDB80sFacmKREZMWZWa2bnmFkqNs19kvDAhYwBqmHImGdm/wTcWGidh6e8ZJQwszpC89qxhPssfwDe6+6tFS2YlEQBQ0RESqImKRERKcm46rRs6tSpPm/evEoXQ0RkzFi+fPlWd28ePOc4Cxjz5s1j2bJllS6GiMiYYWZPDZ4rKFuTlJnNMbPbzOwRM3vYzN5bII+Z2dfNbJWZPWBmp+Stu8TMnojTJeUqp4iIlKacNYwM8EF3vy92cLbczG720C1Bj5cDC+J0GvAtQr8/kwmP2y0ivKiz3Myu99BZmYiIVEDZahjuvsFjT5Duvht4hL4dp0F4u/VHHtwFTDKzGYTuiW929+0xSNxMGORHREQqZESekopdDp9M6Gco3yz69lbZEtOKpRfa92VmtszMlm3ZsmW4iiwiIv2UPWCYWQOhp9P3FXg5xwps4gOk75/ofoW7L3L3Rc3NJd3oFxGRA1DWgBF7i7wW+Km7X1cgSwt9uzeeTeg8r1i6iIhUSDmfkjLg+8Aj7v7lItmuB94Un5Y6HdjloUvum4CzzazJzJoIw1/eVK6yiojI4Mr5lNSZhGEZHzSzFTHtY4RulXH3bxO62D6HMFZxO2HcZdx9u5l9mtD9MMDl7r69XAX908MbmTuljmMPmzB4ZhGRQ1TZAoa730HhexH5eRx4V5F1VxLGIyi7+T9/Ebekz+LY//jWSBxORGRMUl9SwMzENk6aooG3REQGooABZEhiuUyliyEiMqopYAAZUpBTDUNEZCAKGECOJJbrrnQxRERGNQUMIGNJEq4mKRGRgShgEGoYapISERmYAgaQtRQJV5OUiMhAFDAIAcNUwxARGZACBqFJSvcwREQGpoABZC2JKWCIiAxIAQPIWYqEXtwTERmQAgYhYCRVwxARGZACBpBLpDDXTW8RkYEoYABuSdUwREQGoYBBvIeBahgiIgNRwADcUiTVJCUiMqCyDaBkZlcC5wKb3f1ZBdb/G/DGvHIcBzTH0fbWAruBLJBx90XlKieEexhqkhIRGVg5axhXAUuKrXT3/3L3he6+EPgo8Jd+w7C+MK4va7CAcA9DTVIiIgMrW8Bw99uBUsfhXgpcXa6yDMZVwxARGVTF72GYWR2hJnJtXrIDfzKz5WZ2WbnL4Ik0SXLlPoyIyJhWtnsYQ/BK4G/9mqPOdPf1ZjYNuNnMHo01lv3EgHIZwNy5cw+sBIkkKVTDEBEZSMVrGMCF9GuOcvf18XMz8Gvg1GIbu/sV7r7I3Rc1NzcfUAE8kdZTUiIig6howDCzicBZwG/z0urNrLFnHjgbeKisBUkkSemmt4jIgMr5WO3VwGJgqpm1AJ8E0gDu/u2Y7dXAn9x9T96m04Ffm1lP+X7m7n8sVzkh1DAUMEREBla2gOHuS0vIcxXh8dv8tNXASeUpVRGJNEmyuDsxUImISD+j4R5GxVkiScpyZLJ6UkpEpBgFDMCTaQAy3V0VLomIyOilgAFYIrTMdWcUMEREilHAAIg1jGx3d4ULIiIyeilgwL6AkVENQ0SkKAUMIBGbpFTDEBEpTgEDepukMgoYIiLFKGAAiWSoYahJSkSkOAUMIJGqAvRYrYjIQBQwyAsYXZ0VLomIyOilgAEk0tUAZLs6KlwSEZHRSwEDSKZrAMh2K2CIiBSjgAEkq0INI9OtJikRkWIUMIBkbJLKqYYhIlKUAgaQruppktJTUiIixShgAKl9AUNNUiIixShg0BswXE1SIiJFlS1gmNmVZrbZzAqOx21mi81sl5mtiNMn8tYtMbPHzGyVmX2kXGXs0dMk5RnVMEREiilnDeMqYMkgef7q7gvjdDmAmSWBbwIvB44HlprZ8WUsJ+lqBQwRkcGULWC4++3A9gPY9FRglbuvdvcu4Brg/GEtXD/p6lpAAUNEZCCVvodxhpndb2Y3mtkJMW0W8HRenpaYVjZVsUkKdT4oIlJUqoLHvg843N3bzOwc4DfAAsAK5PViOzGzy4DLAObOnXtABbFUeA+DrGoYIiLFVKyG4e6t7t4W528A0mY2lVCjmJOXdTawfoD9XOHui9x9UXNz84EVJpEk4wnIqoYhIlJMxQKGmR1mZhbnT41l2QbcCywws/lmVgVcCFxf7vJ0W1oBQ0RkAGVrkjKzq4HFwFQzawE+CaQB3P3bwOuAd5pZBtgLXOjuDmTM7N3ATUASuNLdHy5XOXt0k8YUMEREiipbwHD3pYOs/wbwjSLrbgBuKEe5ium2NAkFDBGRoir9lNSo0U2aRE4BQ0SkGAWMKKMahojIgBQwoqylSLgChohIMQoYUSZRRSLXXeliiIiMWgoYUc7SJHUPQ0SkqEq+6T2q5Lr3ksype3MRkWJUw4gy9YeR1rchIlKUfiKjXLKGKldfUiIixShgRLlULdUKGCIiRSlgRJ6qpYouQu8kIiLSnwJG5OlaaumkK5urdFFEREYlBYwe6Tpq6WJvZ6bSJRERGZUUMKJEupaEOXs72itdFBGRUUkBI7KqOgA629sqXBIRkdFJASNKrL4NgM69eypcEhGR0UkBI0qd9M8AdHeohiEiUogCRpSsqQegSzUMEZGCyhYwzOxKM9tsZg8VWf9GM3sgTnea2Ul569aa2YNmtsLMlpWrjPnS1SFgdHcoYIiIFFLOGsZVwJIB1q8BznL3E4FPA1f0W/9Cd1/o7ovKVL4+UrGGkenSU1IiIoWUc0zv281s3gDr78xbvAuYXa6ylKKqNgSMXKcChohIIaPlHsZbgRvzlh34k5ktN7PLBtrQzC4zs2VmtmzLli0HXIDqWMPIKmCIiBRU8fEwzOyFhIDx/LzkM919vZlNA242s0fd/fZC27v7FcTmrEWLFh1wR1A1dQ0A5NQkJSJSUEVrGGZ2IvA94Hx339aT7u7r4+dm4NfAqeUuS0+TlCtgiIgUVLGAYWZzgeuAi9398bz0ejNr7JkHzgYKPmk1rOVJhze96VbAEBEppGxNUmZ2NbAYmGpmLcAngTSAu38b+AQwBfhfMwPIxCeipgO/jmkp4Gfu/sdylXOfdG34zOwt+6FERMaicj4ltXSQ9W8D3lYgfTVw0v5blFkiSSdprFsBQ0SkkNHylNSo0Ek1lumodDFEREalQQOGmR1pZtVxfrGZvcfMJpW/aCOvK1GNqUlKRKSgUmoY1wJZMzsK+D4wH/hZWUtVIZlENQkFDBGRgkoJGDl3zwCvBr7q7u8HZpS3WJXRnawjndFTUiIihZQSMLrNbClwCfD7mJYuX5EqpztZT1VOAUNEpJBSAsabgTOAz7r7GjObD/ykvMWqjEy6gdqceqsVESlk0Mdq3X0l8B4AM2sCGt39C+UuWCWs3Z3kGG8nk82RSuoBMhGRfKU8JfVnM5tgZpOB+4EfmNmXy1+0kTdn5nQabC9tnZlKF0VEZNQp5c/oie7eCrwG+IG7Pwd4SXmLVSHVE2hgL7s7FDBERPorJWCkzGwGcAG9N73HpWRNI9WWYXfb7koXRURk1CklYFwO3AQ86e73mtkRwBPlLVZlJOsmAtC+e2eFSyIiMvqUctP7l8Av85ZXA68tZ6EqJV0bXmDvaNtV4ZKIiIw+pdz0nm1mvzazzWa2ycyuNbOKDqdaLlX1oYbR3b6jwiURERl9SmmS+gFwPTATmAX8LqaNOzUNoYbRtUc1DBGR/koJGM3u/gN3z8TpKqC5zOWqiNrGJgCye1srXBIRkdGnlICx1cwuMrNknC4Ctg261RhUHZukFDBERPZXSsB4C+GR2o3ABuB1hO5Cxp/qCQA8uralwgURERl9Bg0Y7r7O3c9z92Z3n+buryK8xDcoM7sy3iwvOCa3BV83s1Vm9oCZnZK37hIzeyJOl5R8RgejuhGABeNytA8RkYNzoB0mfaDEfFcBSwZY/3JgQZwuA74FELsh+SRwGnAq8MnYj1V5pWvoJg2denFPRKS/Aw0YVkomd78d2D5AlvOBH3lwFzApvlX+MuBmd9/u7juAmxk48AybDtKk2p4ZiUOJiIwpBxowfJiOPwt4Om+5JaYVS9+PmV1mZsvMbNmWLVsOukDbmEQip76kRET6K/qmt5ntpnBgMKB2mI5fqKbiA6Tvn+h+BXAFwKJFiw46kKUam5m0aze5nJNIlFSREhE5JBStYbh7o7tPKDA1uvugXYqUqAWYk7c8G1g/QHrZZaubaKKNXXu7R+JwIiJjRqVHCboeeFN8Wup0YJe7byB0dni2mTXFm91nx7Tyq5tMk+1me3vXiBxORGSsGK6aQkFmdjWwGJhqZi2EJ5/SAO7+beAG4BxgFdBOfL/D3beb2aeBe+OuLnf3gW6eD5u7N8L5tPF0WydHNjeMxCFFRMaEsgYMd186yHoH3lVk3ZXAleUo10BesPAYqu+5jl2tO4EpI314EZFRq9JNUqNO9cTQTVb7zoN/4kpEZDwZtIZR5GmpXcAy4INxfIxxo37iNAA6dilgiIjkK6VJ6suEJ5R+Rnjc9ULgMOAxQpPR4nIVrhKqJoQaRmerAoaISL5SmqSWuPt33H23u7fG9x7OcfefA+XvrmOk1YX7Fl2tWytcEBGR0aWUgJEzswvMLBGnC/LWDdcb36NH7WQAsu3jsgd3EZEDVkrAeCNwMbA5ThcDF5lZLfDuMpatMmonkcNItj49eF4RkUPIoPcw4k3tVxZZfcfwFmcUSCRpT0+moaOdTDZHKqkHyUREoIQahpnNNrNfx3EtNpnZtWY2eyQKVymdtdNoZgdb2/S2t4hIj1L+fP4BoQuPmYQeY38X08atXP1hTLcdbGztqHRRRERGjVICRrO7/8DdM3G6Cmguc7kqKjnxMKbZTjbu2lvpooiIjBqlBIytZnaRmSXjdBEwrh8hqmmaxRRa+dIfCo4sKyJySColYLwFuADYCGwAXkfsJHC8qp0ym4Q55x5Z1q62RETGlEEDhruvc/fz3L3Z3ae5+6uA14xA2SrGGg8DYPdWPVorItLjQJ8Z/cCwlmK0iQGje4fG9hYR6XGgAWN8j13aOAOA1J5NdGdzFS6MiMjocKABY/x1CZKvfhrZRBUz2Mz6nXpSSkQEBnjTu0i35hBqF7VlK9FokEjQ1TiHuds3s2brHg6fUl/pEomIVFzRGoa7N7r7hAJTo7uX9PiQmS0xs8fMbJWZfaTA+q+Y2Yo4PW5mO/PWZfPWXX9gp3fgkpPnM9c288SmtpE+tIjIqFS250bNLAl8E3gp0ALca2bXu/vKnjzu/v68/P8fcHLeLva6+8JylW8wVc1HcPiaO/n+hl2VKoKIyKhSzp71TgVWuftqd+8CrgHOHyD/UuDqMpZnaJ64mQbaaVm/odIlEREZFcoZMGYB+S8ytMS0/ZjZ4cB84Na85BozW2Zmd5nZq4odxMwui/mWbdkyjKPknf0ZADo2P6knpUREKG/AKPTobbGnqy4EfuXu2by0ue6+CHgD8FUzO7LQhu5+hbsvcvdFzc3D2MVV0zwADreNrN6yZ/j2KyIyRpUzYLQAc/KWZxPGBi/kQvo1R7n7+vi5Gvgzfe9vlN+Uo3BLsiDRwoqnd4zooUVERqNyBox7gQVmNt/MqghBYb+nnczsGMLY4H/PS2sys+o4PxU4E1jZf9uyStdAsooT7Cn++6bHRvTQIiKjUdkChrtnCEO43gQ8AvzC3R82s8vN7Ly8rEuBa9w9v7nqOGCZmd0P3AZ8If/pqpFiR5/NguQGdndkRvrQIiKjTlm7Y3X3G4Ab+qV9ot/yfxbY7k7g2eUsW0mmHc+clddDZi+bd3cwrbGm0iUSEakYDVg9kGnHYThH2TPc/vjWSpdGRKSiFDAGMu0EAJ5bt5FbHtlU4cKIiFSWAsZAJs8HS/Bi7uH2x7fQmckOvo2IyDilgDGQRBLmnM6zJ3awpyvLXau3V7pEIiIVo4AxmMPPYMLOlUyryfDr+1oqXRoRkYpRwBjMYzdiuQzPTT7Jb+9fz6693ZUukYhIRShgDOYtfwTgEzU/xx1e+T9/rXCBREQqQwFjMDUTYfqzmd7czISaFBt2dejmt4gckhQwSnHkYlh3F0dOdLqzzi/ufXrQTURExhsFjFIcvQRy3Vz3sk4aa1J86ncr2dbWWelSiYiMKAWMUsw5DWomYo/fxHXvfB7ZnPOyr95e6VKJiIwoBYxSJNNw7Lmw8noWNCWZMbGGrW1d/HbFM5UumYjIiFHAKNXCN0DXbvjOWfzlwy+ksTrF+3++goee0ZjfInJoUMAo1dznwaTDYcIM0skEt35ocXzM9g4e27i70qUTESk7BYxSJRJw8sWw5i+w5XGaG6u59UOLSSWNc77+V/7y+DCOJy4iMgopYAzFcy4FDH74SgDmT63n1g8upiqZ4JIr7+Hbf3mSbK7YsOUiImObAsZQNDTDKW+Cvdthx1MAzJlcx7KPv4SmujRfuPFRTvzUTfxjncYAF5Hxp6wBw8yWmNljZrbKzD5SYP2lZrbFzFbE6W156y4xsyfidEk5yzkkZ/07WAJu/cy+pPrqFPf9x0v52oULae/M8ur/vZOLv383965V77YiMn5Y36G0h3HHZkngceClQAtwL7A0f2xuM7sUWOTu7+637WRgGbAIcGA58Bx3H/BP90WLFvmyZcuG8zQK+/IJ0NoCb78VZj2nz6o9nRl+ctdTfP7GRwE4/YjJ/OviozjzqKkkE1b+somIDIGZLXf3RaXkLWcN41Rglbuvdvcu4Brg/BK3fRlws7tvj0HiZmBJmco5dP/6d0hWwVWvhGzf3mvrq1P8y1lH8sjlS/iPc4/n7tXbedOV93DUx27gP37zEH9/cpvuc4jImFTOgDELyO90qSWm9fdaM3vAzH5lZnOGuC1mdpmZLTOzZVu2jNCTSjUT4LXfh+498PVTCmaprUry1ufP55FPL+EbbziZpro0P77rKZZ+9y6O+tgNfPw3D3LTwxvVXbqIjBmpMu67UPtL/z+tfwdc7e6dZvYO4IfAi0rcNiS6XwFcAaFJ6sCLO0THnwcnvh4e+Dk8+gc49hUFs9Wkk5x74kzOPXEm7V0Zbnt0C394cD0/uWsdP7lrHQD11UkuOWMeZx41lVPmNlFblRyx0xARKVU572GcAfynu78sLn8UwN0/XyR/Etju7hPNbCmw2N3/Ja77DvBnd796oGOO2D2MHt174b+Ogu52ePttMHNhyZt2ZXL8Y90O/vbkNq68Yw1tnZl96xqqU1z43DmcPLeJhXMnMXNiDWa6/yEiw28o9zDKGTBShJveLwaeIdz0foO7P5yXZ4a7b4jzrwb+3d1Pjze9lwM97T33EW56D/jY0YgHDIDdG+F7L4GuNnjT9TDjxAPaTVtnhnvXbufu1du5Z8027lu3c986A15y/HROmDmB42dM4PiZE5g1qVZBREQO2lACRtmapNw9Y2bvBm4CksCV7v6wmV0OLHP364H3mNl5QAbYDlwat91uZp8mBBmAywcLFhXTeBhc8rvwMt+PzjvgoNFQneKFx0zjhcdMA0IN5NGNrax4eicr1u3k/pad3Lxy0778yYRx6rzJHD9zQggkMydwZHMD6aRerRGR8ihbDaMSKlLD6LF9DXzzNMhl4LXfhWe9dtgP0d6V4dGNu1m5vpWH17dy/f3PsKez7+h/z541keNnTOCEWaE2cuyMCTRUl/NWlYiMZaOiSaoSKhowAHY9A/97GnTuhkVvhZd9DtI1ZT1kJptjzdY9rNzQui+Q/G3V1j5PCMyaVMuC6Q0smNbAgmmNHDmtgaOaG5hYly5r2URk9FPAqKRsN9zyKbjzf6CqPtwMbz5mRIvg7mxs7WDl+lYe2dDKE5vbeHxTG49saO2TL5UwFs6ZxBHN9cyf2sD8qfUc0VzP4VPqqE7pSS2RQ4ECxmjw6A3wi4shl4XT3wlnfRhqmypapGzOeXp7O6s2t/HkljbWbN3D6q17uHfN9v2eWZ4zuZb5Uxs4IgaR+VPDNHNiLQm9sS4ybihgjBZtW+C2z8DyH0LdZHjRx+GUSyAx+v56b+3oZu3WPazZuocnt4TPWx7ZRHtXdr+8xx7WyPyp9cydUsfsSbXMbqpjVlMtsybVUq/7JSJjigLGaLPhAbjqFdDZCtOfBUs+D/NfUOlSlcTd2by7k9UxiKzOq5ms2bpnv/xNdWlmNdUye1JvEJndVBvSmuqYWKv7JiKjiQLGaOQOK38L110G2U446qXw/PfB4WfCGH2fIpdztrR10rJjLy3hl02HAAAT10lEQVQ72nlm515aduzlmR17uXvNNjq6c/tt01idisGjJ5jU9VmeXF+l90tERpACxmjWvRfu+hbc9jnIdcPMU+D0f4Xjz4dUVaVLN6zcne17ukIQ2RmDyr75vTxaYGjb2nRyv5pJT2CZ3VTL1IZq9forMowUMMaCrna4/2dw08chszf0fvuCfwuj+jVMq3TpRoS707o3Q8vOEEj6BJadoaayo71v54wJgykN1Uxr7JlqmDYhzE9pqGZyfRVTG6qYXF/NpNq0btCLDEIBYyzJ5eDJW+Hub8OqmwGDky6E574dZp0yZpurhsuezkxeEOlgc2sHm1s72by7g827O9myu5OtbZ0U6jE+YdBUV8WUhiom11cxpT4ElCkNVUypD0GlN8BUMamuSrUXOeQoYIxVW5+Ae64IE8DUo0PwOPH1MHF2Zcs2imVzzrY9nWxr62L7ni627elie1sn2/bNd4X1e8L6ne2Fu5RPGEyq6wkmfQNNofkmBRgZBxQwxrqOVnj413D/NbDuzpA2/yxY+AY49lyobqhs+ca47myOHe0heGxv62JrDDA9waY38IS0/s1iPczCTfwJtWkm1qaZUJOmsSYsT6hJM6E2FT9jek2ahuoU9dXJ+Jmiriqpm/xSUQoY48n2NWHMjTu+ApmOkHbS0jDN+ydIqLPBcstkc+xo794XRPJrMq17u2nd282uvd3s7sjQ2hGWWzsyfbqsL8YM6qtCEKmvTtEQp/p9nzG9Kj8tpDfWxPmq3vSqlP49yNAoYIxH7rDuLrj/arjvhyEtWQ1nvAtOvACajz3k73eMNplsjrbODK17ewNJW2eGPV0Z2jqz7OnMsKczBJYwn90335aXvrO9u/DoYQVUJRN9gk9+EKqrSlKTTlKTTlCdCp816STV6STVqTBfk0pQnf+Zlzf/U01x44cCxnjXvRceuwFWXB1vlANTjoLjXgnHnQczT1bwGGe6Mrne4NLVE1RC0Gnr6A0ubV37B5/eoJSlM5OloztHRybLwfzXTyeN6lSSqlSCqmQifPabTyeNdDJBKhHmU8kE6YSRiulhXUxPGqlEglTSqEqGz1Rcn0zYvs/e+USfNDNIWphPJKx33nrysG8+Yb15Eoni2yWMuO/x/X9JAeNQsnsTPPo7eOR3sPrPIW3iHDh6SZjmPb/sPebK2OPudGedjkyWju4snd25fcGk6Gd3lo5Mjs4YcDq6s3RlcmHK5ujOhvnOmJbJOd3ZHN1ZJ5PtXc5ke9JDWibrdOdyBxXAyi1hxCASgpPlLxOXE2G+N0+/ZWJa3vY969m3PzCs4N97/QNX/m/3pLo011x2xgGd26gYQElGSON0eO7bwtS+HR67MQSPZd+He78L6bpww/zos2HBy2DirEqXWEYBM6MqZVSlEkyoGR3dtWRzeYEkBpFcDjK5HNmck8k52bwpBJuwLueQ87jOnVzME9LYL61Yev72PfvMOXHZcUKae/jB7smTv+zEz5jmPXnISyM/f1jPvuP5fsGzfyx1Z18QApgwQl3uKGCMJ3WT4eQ3hqm7A9beAY//EZ64CR6/MeSZ/ixYcDYc+SKYcyqkqitbZpEoNB2F+ywyOpW1ScrMlgBfIwzR+j13/0K/9R8A3kYYonUL8BZ3fyquywIPxqzr3P28wY53SDZJlcIdtjwWA8ef4Kk7Qnq6LvRldeSL4MgX6sa5yCFoVNzDMLMk8DjwUqCFMD73UndfmZfnhcDd7t5uZu8EFrv76+O6Nncf0gsHChgl6tgVah9P3garb4Ntq0J64ww4YnEIIEcsPmS6KBE5lI2WexinAqvcfXUs1DXA+cC+gOHut+Xlvwu4qIzlkR41E+HYV4QJYOe63uBx/zXh0V2AaceHGsjhZ8DcM2DCzMqVWUQqrpwBYxbwdN5yC3DaAPnfCtyYt1xjZssIzVVfcPffFNrIzC4DLgOYO3fuQRX4kDVpLjznkjC9Ngcb7w8BZO1fw43ze78b8x0eAkdPAJl6tJqwRA4h5QwYhX5JCrZ/mdlFwCLgrLzkue6+3syOAG41swfd/cn9duh+BXAFhCapgy/2IS6RCO9xzDwZ/ukDkM3Apgfhqb/Dur/DQ9fCA9eEvLWTQ+CYezoc/jyYcRIkR8cTNyIy/MoZMFqAOXnLs4H1/TOZ2UuA/wOc5e6dPenuvj5+rjazPwMnA/sFDCmzZKo3gJzxr+EG+vbV8NSd4c3zdXfCY3/ozT/vn3prIbOeE5q/RGRcKGfAuBdYYGbzgWeAC4E35Gcws5OB7wBL3H1zXnoT0O7unWY2FTgT+FIZyyqlMoMpR4bplItD2u5Nofax7q7weXvepZp6DMxeFKZZi8J9kaSe5hYZi8r2P9fdM2b2buAmwmO1V7r7w2Z2ObDM3a8H/gtoAH4Z32LseXz2OOA7ZpYDEoR7GCsLHkgqr3E6nPCqMAF07oaWe6FlOTyzLLwLsuKnYV26DmYshNnPCbWWGQuhab46URQZA9Q1iJSfO+xYC88sj4FkGWx8ALJdYX31hHD/Y8ZJcNizYfoJ4Ya6XioUKbvR8litSGAGk+eH6dmvC2mZLtjyCGy4H9avgA0r4J7vQjbexkqkQtCYfkJ4O336s8J842F6MkukQhQwpDJSVb21ilPeFNKyGdj+JGx6CDY9HKZ1d8GDv+zdrnZyeCN92rHhs/kYaD4uvGSoQCJSVgoYMnokUzEAHAPPem1v+t4dsGllDCIPhW5OHro2vLHeI5EKN9Wbj4YpC2DqgtDle9M8PeorMkwUMGT0q22CeWeGqYc7tG0OzVpbHoMtj8LmR+HRG6B9a28+S4ag0RNAphwFk48I04RZutkuMgQKGDI2mYWnsxqnh36v8u3dAduehK1PwLYnQl9ZW1eF8UJ6hrkFSFaFYNI0PwaR+b3BZOKc0GwmIvsoYMj4U9vU++5HvlwOWlvCOOnbV4dpx5qwvPYO6N7Tm9cS0DgzdJvSZ5oTOmlsPCw83aX7JnIIUcCQQ0ci0fvDf8RZfdf1NHHt6Akma2DX06Fjxqf+Bg/+AjzXd5t0XQgcPQGkcQY0TO+73HgYVA+p02WRUUsBQwT6NnHNPX3/9dluaF0fgsjujbB7Q/yM0/oVsP3awvuuaowBJH/qF1QaDoOquvKeo8hBUsAQKUUyDU2Hh6kY9/CWe35AadvYd/mR3/W9j5KvemLxgNIzNRymMdqlYhQwRIaLGdRMCFPz0cXzuUPHztAH176aSr/Pp+6EXesKb1/bFAJJ3ZQwLG/dlPB+St3kAp9NUDNJT4PJsFDAEBlpZuGHvLYpvIBYjHt44mv3hgKBZSPs2QqbH4H27SGfZ4scLxGPVyCY7FueEoPLhHAzvzoGPr3DInkUMERGK7NYg5gcukUZSC4Hna3Qvi0Ej/btsHd7gc9tsKsl9OXVvh0yewfeb6o2BpHG3iCS/9lnvjHOT+yblq7V02TjhAKGyHiQSEDtpDANRffe3mCydwd0tIbA09Ea7sd07upN69wd5ndv7F3ftbuEsqXyAkvj/gGlZgJU1UNVQ3jyrKoO0vUxrWe+rnd9qloBqEIUMEQOZelamDgrTAcil42BJS+g7As4/eb3BaFW2Pl0CEY92xRrTivEEnlBpL53Ph2Xq+r7zqdqwnmmqsP8vs/8qbpAnlqN3dKPvg0ROXCJ5IHVbPK5Q6YTutuhqw262sNLlF3t0LWnd77P+v7ze0LwadsUt2nv/TwYlhwgoPQLOKma8ARb0fXVkKwO94WSVXHKm08VSNs3Xx2+6wrXrBQwRKSyzMIPbbom3K8ZTrlc6DI/0xGD0t7w2bOc6be83/qOflO/PF17Qt9lffYR82Y7By/fkNj+gaQnyDRMhzffMMzH258ChoiMX4kEJGpD7WCk5XJhkLD8YJPtDmnZrjjfmTff1Xe+T/5BtquqH5FTKmvAMLMlwNcIQ7R+z92/0G99NfAj4DnANuD17r42rvso8FYgC7zH3W8qZ1lFRIZVIgGJmnH1omXZ3uYxsyTwTeDlwPHAUjM7vl+2twI73P0o4CvAF+O2xwMXAicAS4D/jfsTEZEKKefrn6cCq9x9tbt3AdcA5/fLcz7wwzj/K+DFZmYx/Rp373T3NcCquD8REamQcgaMWcDTecstMa1gHnfPALuAKSVuC4CZXWZmy8xs2ZYtW4ap6CIi0l85A0ah57+8xDylbBsS3a9w90Xuvqi5uXmIRRQRkVKVM2C0AHPylmcD64vlMbMUMBHYXuK2IiIygsoZMO4FFpjZfDOrItzEvr5fnuuBS+L864Bb3d1j+oVmVm1m84EFwD1lLKuIiAyibI/VunvGzN4N3ER4rPZKd3/YzC4Hlrn79cD3gR+b2SpCzeLCuO3DZvYLYCWQAd7lPpS+A0REZLhZ+IN+fFi0aJEvW7as0sUQERkzzGy5uy8qKe94ChhmtgV46gA3nwpsHcbijAU650ODznn8O5jzPdzdS3piaFwFjINhZstKjbLjhc750KBzHv9G6nw1bqOIiJREAUNEREqigNHrikoXoAJ0zocGnfP4NyLnq3sYIiJSEtUwRESkJAoYIiJSkkM+YJjZEjN7zMxWmdlHKl2e4WJmc8zsNjN7xMweNrP3xvTJZnazmT0RP5tiupnZ1+P38ICZnVLZMzhwZpY0s3+Y2e/j8nwzuzue889jVzXErmd+Hs/5bjObV8lyHygzm2RmvzKzR+P1PmO8X2cze3/8d/2QmV1tZjXj7Tqb2ZVmttnMHspLG/J1NbNLYv4nzOySQscq1SEdMEoc5GmsygAfdPfjgNOBd8Vz+whwi7svAG6JyxC+gwVxugz41sgXedi8F3gkb/mLwFfiOe8gDNwFRQbwGoO+BvzR3Y8FTiKc+7i9zmY2C3gPsMjdn0XoeuhCxt91voowgFy+IV1XM5sMfBI4jTCm0Cd7gswBcfdDdgLOAG7KW/4o8NFKl6tM5/pb4KXAY8CMmDYDeCzOfwdYmpd/X76xNBF6Nr4FeBHwe0JX+VuBVP9rTujn7Iw4n4r5rNLnMMTznQCs6V/u8Xyd6R0vZ3K8br8HXjYerzMwD3joQK8rsBT4Tl56n3xDnQ7pGgZDGKhpLItV8JOBu4Hp7r4BIH5Oi9nGy3fxVeDDQC4uTwF2ehigC/qeV7EBvMaSI4AtwA9iM9z3zKyecXyd3f0Z4L+BdcAGwnVbzvi+zj2Gel2H9Xof6gGj5IGaxiozawCuBd7n7q0DZS2QNqa+CzM7F9js7svzkwtk9RLWjRUp4BTgW+5+MrCH3maKQsb8OccmlfOB+cBMoJ7QJNPfeLrOgznowehKcagHjHE9UJOZpQnB4qfufl1M3mRmM+L6GcDmmD4evoszgfPMbC1hDPkXEWock+IAXdD3vIoN4DWWtAAt7n53XP4VIYCM5+v8EmCNu29x927gOuB5jO/r3GOo13VYr/ehHjBKGeRpTDIzI4w38oi7fzlvVf6gVZcQ7m30pL8pPm1xOrCrp+o7Vrj7R919trvPI1zLW939jcBthAG6YP9zLjSA15jh7huBp83smJj0YsI4MuP2OhOaok43s7r477znnMftdc4z1Ot6E3C2mTXFmtnZMe3AVPqmTqUn4BzgceBJ4P9UujzDeF7PJ1Q9HwBWxOkcQtvtLcAT8XNyzG+EJ8aeBB4kPIFS8fM4iPNfDPw+zh9BGLFxFfBLoDqm18TlVXH9EZUu9wGe60JgWbzWvwGaxvt1Bj4FPAo8BPwYqB5v1xm4mnCPpptQU3jrgVxX4C3x3FcBbz6YMqlrEBERKcmh3iQlIiIlUsAQEZGSKGCIiEhJFDBERKQkChgiIlISBQwRESmJAobIQTKzhWZ2Tt7yeTZMXeWb2fvMrG449iVysPQehshBMrNLCS9KvbsM+14b9711CNsk3T073GURUQ1DDhlmNi8OMPTdOPjOn8ystkjeI83sj2a23Mz+ambHxvR/joP23G9mt8cuZS4HXm9mK8zs9WZ2qZl9I+a/ysy+ZWEwq9VmdlYcGOcRM7sq73jfMrNlsVyfimnvIXSud5uZ3RbTlprZg7EMX8zbvs3MLjezu4EzzOwLZrYyDqbz3+X5RuWQU+nX3zVpGqmJMLZABlgYl38BXFQk7y3Agjh/GqH/IQjdLsyK85Pi56XAN/K23bdMGATnGkLXDecDrcCzCX+sLc8rS08XD0ngz8CJcXktMDXOzyT0o9RM6KX2VuBVcZ0DF/TsizAeguWXU5Omg51Uw5BDzRp3XxHnlxOCSB+xS/jnAb80sxWEQWdmxNV/A64ys7cTftxL8Tt3d0Kw2eTuD7p7Dng47/gXmNl9wD+AEwgjQPb3XODPHnppzQA/BV4Q12UJPRNDCEodwPfM7DVAe4nlFBlQavAsIuNKZ958FijUJJUgDMazsP8Kd3+HmZ0GvAJYYWb75RngmLl+x88BKTObD3wIeK6774hNVTUF9lNobIMeHR7vW7h7xsxOJfTieiHwbkJX7yIHRTUMkX48DDS1xsz+GUJX8WZ2Upw/0t3vdvdPEIb6nAPsBhoP4pATCAMf7TKz6fQdDCh/33cDZ5nZ1Dge/VLgL/13FmtIE939BuB9hN5sRQ6aahgihb0R+JaZfRxIE+5D3A/8l5ktIPy1f0tMWwd8JDZffX6oB3L3+83sH4QmqtWEZq8eVwA3mtkGd3+hmX2UMO6DATe4+2/33yONwG/NrCbme/9QyyRSiB6rFRGRkqhJSkRESqImKTmkmdk3CWOB5/uau/+gEuURGc3UJCUiIiVRk5SIiJREAUNEREqigCEiIiVRwBARkZL8P/hPLQ0xaN0OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x241990e5ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#调整max_depth和min_child_weight之后再次调整n_estimators(6,4)\n",
    "xgb2_3 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #数值大没关系，cv会自动返回合适的n_estimators\n",
    "        max_depth=9,\n",
    "        min_child_weight=0,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel=0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb2_3, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1EAAANGCAYAAAAcVgSJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XmYXGWZ///3TWInpEM3SFhkzSgMGjdWFXdlEBXXGUdFURnR+SluDKISfgwiAkFHQdBRvn51YEDH0XFBERUkwgwCLoAoyK4mbCMQIN0JIWmW5/vHOW1VV6q7T1VX16nl/bquunLOqVOn7uru5OpPnue5T6SUkCRJkiQVs0nZBUiSJElSNzFESZIkSVIDDFGSJEmS1ABDlCRJkiQ1wBAlSZIkSQ0wREmSJElSAwxRkiRJktQAQ5QkSZIkNcAQJUmSJEkNMERJ6gkR8W8RsSEinl7nuaMiIkXEq2uOD+XP/TIiVkfEwxFxd0T8JCLeEhHzqs5dnF+j+jEaEb+NiMMjYk47PudUIuKwiDik7DpmIiIWRMRxEfHiOs8dkn/dF5dQ13Z5Xbu3+73LFBFnRcTasusAyL/+KSIWlV2LJBmiJPWKw4E/A/8eEY8bP5iHqk8AZ6WUzqs6vivwG+D/B34OvB14KfAB4E7g34Bj6rzP54F988cbgcuAU4FPt/4jNeww4JCyi5ihBcDHgRfXee58sq/7/7azoNx2ZHX1VYiSJNU3t+wCJKkVUkqjEXEocCFZ+Pl4HqbOAe4mC1kARMRc4Fzg8cCzUko31FzuWxFxPLBHnbe6LaX0i6r9n0TE04CDgA+37ANpIymle4F7y66jlSJiQUppXdl1SJIa40iUpJ6RUroIOAM4OiL2Ao4DngkcmlIaqTr19cAS4MQ6AWr8WitTSucWfOsR4OHqAxGxSUR8NCJuzKcZ3hMRZ0fEDrUvjoh35tMC10fE/RHxvYh4Ss05T4yI/4yIu/Lr3R0Ry8enl0XECuCpwIuqphuumKro/JwvRMTbIuKGiFiX1/Gqgp+7+loDEXFM1ee9NyLOjIitas57aURcEhH3RcRDEXFbRHwnn8a3mEpI+njV5zgrf+1G0/nya10XEftGxOX5NVdExD/kzx8YEVfnn+3aiHh5TT275HXekp9zZ0ScVz0tNJ9a+Ot898yquo6rOuc1EXFFfo01EfHTiNi35r3Gp6PtGRHfjogHgD/kz035/Z3ka354fr1d6jz3qYgYG5/6FhF7RMQP85/DDfn7nF/v57EZEfH8vN41+dfg8og4cJLzrsh/1u+MiE9GxLtqv68zrOVpEfH9iHggf59rIuIdNedskv+83pT/zKyOiN9FxIeqztkqIr4cEbdX/UxfFhF/04o6JXU3R6Ik9ZqPAAcA3wZ2BM5IKf205pz98z9/0MT1N4lsJAtgGHgt8HLgUzXnfQn4R+ALwA+BxcAngRdHxJ4ppVUAEbEUOAn4BrAU2JIs/F0REfuklG7Jr/cjYA7wUeA2YBHwXGDz/PnX5595hGxaH8CGAp/nQGAf4FhgbX7970XEbimlPxZ4PRGxCfB94AVk0xovB3Ymm0Z5SUTsnVJ6KP8l+XzgUuCdwGpge7Kv3wDZNL2XAz8Bvgp8JX+L6UaftgXOzN/7DrIpmf8WETsCbyD7+o7kn/HciHhiSumu/LXbAfcBR+Xv83jgHcAvI2KPlNJNwNXAP+TvcUL+Gcjfi4h4C/B1slHQg4B5+dfxkojYL6X085p6vwv8J1ngH8yPTff9redrZD93h1A19TSy9XkHA+ellFZFxCDwU+BPwPvIRma3BV4CbDbF9QuJiBfl1/8dcCjZz91hwHkRcVBK6Zv5ec/Iz7uZ7Gu8DnhPXmtLRMRuZD9/9wAfJPveHgycFRHbpJTGp91+lOzv2QnA/wCPA57MxK/3OcCeZFN+b86f25Ps76ikfpdS8uHDh4+eepD9IpvIfilfWOf5H+fPz6s5HmT/uTT+mFP13OL8NfUeZ9ac++T8+L/WXP9Z+fET8/3NyX6RPL/mvB2B9cDX8/0t89d9aJrPfR1wSQNfp0S2jmyzqmPbAI8CRzVwnTfn1/rbmuN758ffm+//Xb7/zCmutSg/57g6zx2SP7e46tgl+bG9qo49Hngk/9puV3X8mfm5H5ji/eeQ/UJ9M3BKnc9ySM35m5CtofsdsEnV8YVkYeWyqmPH5df4RM01Cn1/J6n3O8DtNe/9ivx6r8r398r3X9vE9c8C1k5zzhX5Z11YdWwOcG1eW+THvkUW1BfVfP1+X/t9neR9xr9+i6Y45xv5350da47/CHgQGM73zwN+M837rQFObfRr5sOHj/54OJ1PUk/JR0U+ADwGbE32i3NRHyKbljf++G2dc04jG7nZh+x/8o8mazDxjapzXpL/eVb1C1NKvwJuAPbLD+0LbFrnvNuBn1Wddz/ZtK+PRMQR+dSsVv37fXFKaU3Ve99N9r/4OzdwjVeRjSqdFxFzxx/ANWQh7cX5edcAY8CXI+IdEfHEVnwA4H9TSleN76SU7if7DNekyogTZF97qPpsea1HR8T1ETFGFr7GgF2BCVMqJ7Eb2WjWOSmlx6pqWEsWcJ4TEQtqXvOdmv2ZfH/PBHYAqqeY/QPZ1/3H+f6twAPApyLiPRGxpOC1p5WPcj0b+Hb+mQFIKT1KNpKzA9nXCOBFwM9SPgqbn/cYWbhqlZcCy/O/Q9XOImtaMj7F8lfAMyPiixFxQEQM1bnWr4BD8ml/z4mqhjWSZIiS1GuOJPtF6S3ALWTTujatOee2/M/aoPAfVALS1ZNc/46U0pX545KU0jKyaXp/HxEH5OeMT/ep10XurqrnC52XUkpkgeoCsmlIVwP3RsTpETHT6Vj31Tm2gSzcFbUN2ajaGBND6MNk08YWAaSU/kD2y/49wL8Cf4iIP1SvQ2nS/XWOjdUeTymN5Zvzqw6fQvb9Oxd4NVkg2IcsQBf5Gkz3PdwE2KLm+IRzZ/j9/XF+vfE1YFsArwHOzoMMKVsP+CKyEHsS8Pt8TdQnWhAMtiAbwZ3s88PEn/e765xX71iztixYyzKyfyueQ/Y1vC9f07V31WveBPw78C6y0bb7I1vXuG0L65XUpQxRknpG/j/sx5P9AvlNsulfuwAn1pw6vkbqNdUHU0r3jAcksqk8Rf0u/3N81Gs8mDyhzrnbAasaPI+UNbo4NKW0Ldn/7J9Ktu7kXxqoc7asIvss+0zyGF+jRUrp0pTSq8nWkz2H7JfTz0XEm9tddO5gsp+Xo1NKF6SUfpV//4vei2i67+FjZKNA1VLtic1+f6tGfF4XEZuT/efBPLIRqurzrk0pvZksROwOfJNsjdhMO0o+QPYZJ/v8MPHnfZs657UylNxXpJaU0iMppVNSSnuSTf88iGwa7QXjI4cppVUppcNTSovJ/sNlKfC31IwcS+pPhihJPSGfPvbvZL8kfQggZa3ITwE+FBHPqzr9e8D1ZF38ntyCtx/voHZP/ufP8j8nLJiPiH3Ipogtzw9dATxU57wdyKcl1XuzlNLNKaUTyNac7Fn1VKMjSK3yQ7JfzudUjdJVP26qfUFK6dGU0i/JGh1A5XOMN8No1+dI1DTgyLvKbV9z3mR13US2JuotERFV1xgkWwN2RWqwhfkU39/JnEk2unYQ2X8cXJFSunGSa6eU0m9TSv9ENgWzyPWnqvVB4JfA31aP+ObTEQ8ma75xc374v4GXRtXNcvPz/n4mNdRYnr/HdjXH3062Ru4XtS9IKa1OKX2bbHT08WTrH2vPuS2l9AWy/4CZ0ddMUm+wO5+kXrGUbPH/K1JKq6uO/zPZNK1/i4jdU0oPpZQejYjXkU2f+lVE/F+yBgUPkE1LezbZqFK99uc7RcRz8u1BsqmDS4GVZF3XSCndFBFfBj4QEY+RTRdaTDZt7HayUQZSSqsj4pPASRFxNtm6qi3Jbuq6nqy73XhXsy8A/0U2RXGMLGQ9Azi5qrZrgTdHxJuAPwLrU0rXNvJFbNJ/Am8FfhQRp5GtJXmYbD3MS4Dvp5S+FxHvyes+n2xK5XyyLn0AFwGklNZExErgtRGxnGxK3qqU0opZqv2HZOtebiQbUdyLrMPjHTXn/YEs8L41Im4ga5BwV0rproj4KFl3vh9GxP8hGwn6CNnP0lHTFdDA97eulNKNEXEF2c/hjmRdIauv/yqyUa1zyX4ugmxEZXMqo7JTmRMRb6hz/MGU0o/z9/0pcHFEfCav/zDgacBB+XRFyEaEXw0sj4gTyb6e76HSofAxinl1RGw0UpwHoU+QrdG7OLJ7vd1P9rN5IPDRfGojEXEeWSOWK8m6Mu5Mdi+5lcAtETEMXEw2xfdGspHpfci6R363YJ2SelnZnS18+PDhY6YPssAzBnx5kuefQ9Zx7pSa40NkvwD+isq9nu4ma1V9GLCg6tzFbNyV7yGykYhTgW1rrr0J2fqWm/La7iVfaF+nvkPJ1uBsIBsdOBdYUvX81mSjDeO/vK/Jzz+ciV0BdyYLhqN5fSum+bol4At1jq8AzmrwezCXbGrYNfnXZU1e7xnALlXfh+/m119PNmp4CfDqmmvtR7YuaH1e41n58UOo353vukk+ww+n+8xkQeIr+ff9QbL268/Pr3tJzWvfnH+mMWo6CJK1uv9F/tnXkoXC59a8/jjqdJcr+v2d5uv/7vza64Chmud2IwsDt+bPryYbPXpHgeuexeRdKVdUnfd8slGgtfl7XEHeHbDmes/Pv07rydYufZrs70ki75w3RS3HTVFLqjrvaWS3L1hN9nfqGjbuqngEcBnZ38sNZOHpK8DO+fPzyG5T8FuyfxvWkYWp46j6d8GHDx/9+xhvOypJktR2EXEhWTD+67JrkaSinM4nSZLaIiJOAX5DNq318WRT7fYnG42VpK5hiJIkTSpv2DGVx1LV/ZGkacwh66C5Ldk0vOuBt6WUvlZqVZLUIKfzSZLqiojFwJ+mOe0TKaXjZr0YSZI6iCNRkqTJ3EXWkWy6cyRJ6iuOREmSJElSA7zZriRJkiQ1oO+m8+V3lN+O7D4ckiRJkvrbZmQ3UC88Ra/vQhRZgKq9E70kSZKk/rUDcGfRk/sxRK0BuP322xkaGiq7FkmSJEklGR0dZccdd4QGZ6n1Y4gCYGhoyBAlSZIkqWE2lpAkSZKkBhiiJEmSJKkBhihJkiRJaoAhSpIkSZIaYIiSJEmSpAYYoiRJkiSpAYYoSZIkSWqAIUqSJEmSGlB6iIqIwyLiTxGxPiKuiogXTHHuIRGR6jzmt7NmSZIkSf2r1BAVEW8CPgecCOwBXAr8OCJ2muJlo8ATqh8ppfWzXaskSZIkQfkjUUcAX00pfSWldENK6XDgduC9U7wmpZT+XP1oT6mSJEmSVGKIiogBYC/gwpqnLgSeO8VLF0bEyoi4IyJ+GBF7TPM+8yJiaPwBbDazyiVJkiT1szJHohYBc4C7a47fDWw7yWtuBA4BXgMcBKwHLouIXad4n6XASNXjjuZLliRJktTvyp7OB5Bq9qPOsezElH6RUvpaSum3KaVLgTcCNwMfmOL6y4DhqscOMy9ZkiRJUr+aW+J7rwIeZeNRp63ZeHSqrpTSYxHxa2DSkaiU0gZgw/h+RDReqSRJkiTlShuJSimNAVcB+9c8tT9weZFrRJaIdgf+t7XVSZIkSVJ9ZY5EAZwCnBMRVwJXAP8I7AScARARZwN3ppSW5vsfB34B3AIMAR8kC1Hva3/pkiRJkvpRqSEqpfTNiNgSOJbsnk/XAa9MKa3MT9kJeKzqJZsDXyabAjgC/AZ4YUrpV+2rWpIkSVI/i5Tq9nDoWXmb85GRkRGGhobKLkeSJElSSUZHRxkeHgYYTimNFn1dJ3TnkyRJkqSuYYiSJEmSpAYYoiRJkiSpAYYoSZIkSWqAIUqSJEmSGmCIkiRJkqQGGKIkSZIkqQGGKEmSJElqgCFKkiRJkhpgiJIkSZKkBhiiJEmSJKkBhihJkiRJaoAhqkTrxh5h8VHns/io81k39kjZ5UiSJEkqwBBVourgZIiSJEmSuoMhqkPsfcJyg5QkSZLUBQxRJTrtolsm7C859gKDlCRJktThDFEl2n6LTcsuQZIkSVKDDFEl+rs9t9/omKNRkiRJUmczRJUoIuoeN0RJkiRJncsQ1YFsMiFJkiR1LkNUiRYMzJ30Oaf1SZIkSZ3JEFWiBQNzuf74AyZ93iAlSZIkdR5DVIczSEmSJEmdxRBVsulGo8BGE5IkSVInMUR1gAUDc7nymP0mfd5GE5IkSVLnMER1iKmaTIDT+iRJkqROYYjqEEWm9RmkJEmSpPIZojqI66MkSZKkzmeI6jBFgpQkSZKk8hiiOtBUjSZsMiFJkiSVyxDVoaZqNOHaKEmSJKk8hqgONd20PoOUJEmSVA5DVAczSEmSJEmdxxDV4aa7Ea8hSpIkSWovQ1QXmGp9lI0mJEmSpPYyRPUAp/VJkiRJ7WOI6gJF7h1lkJIkSZLawxDVJQxSkiRJUmcwRHWRIkHKECVJkiTNLkNUl5muW9/eJyxn8VHnG6YkSZKkWWKI6kJTdesb59Q+SZIkaXYYorpQkWl94NQ+SZIkaTYYorpUkSDlPaQkSZKk1jNEdTE79kmSJEntZ4jqctM1mgCDlCRJktRKhqgeYKMJSZIkqX0MUT3ARhOSJElS+xiieoSNJiRJkqT2MET1ENdHSZIkSbPPENVjFi2cb8c+SZIkaRYZovqUQUqSJElqjiGqB9loQpIkSZo9hqgeZaMJSZIkaXYYonqYjSYkSZKk1jNE9ThvxCtJkiS1liGqx7k+SpIkSWotQ1QfWDAwlxUnHzhlmHJ9lCRJklSMIaqPTLdGyml9kiRJ0vQMUX1mujVSBilJkiRpaoaoPlNkjZRBSpIkSZqcIaoPFQlShihJkiSpPkNUn5pufZSNJiRJkqT6DFF9rMj6qMVHnW+YkiRJkqoYovqY95CSJEmSGmeI6nNFgpRT+yRJkqQKQ5Ts2CdJkiQ1wBAlYPpGE2CQkiRJksAQpSrTNZqALEitWru+DdVIkiRJnckQpb9YMDCXFScf6BopSZIkaQqGKG3EqX2SJEnS5AxRqmvRwvk2m5AkSZLqMERpUnbtkyRJkjZmiNKUigQpQ5QkSZL6iSFK05pujZSNJiRJktRPDFEqZLr2507rkyRJUr8wRKmQIu3PvYeUJEmS+oEhSg2Zbo2UU/skSZLU6wxRath0a6Sc2idJkqReZohSU4qskVp81PmGKUmSJPUcQ5SaUmSNlCRJktSLDFGaEaf2SZIkqd8YojRjtj+XJElSPzFEacam69gHBilJkiT1DkOUWqLofaQMUpIkSep2hii1lGukJEmS1OsMUWo5259LkiSplxmi1HJF1kgBhihJkiR1JUOUZkWRILX3CcsNUpIkSeo6hijNmqJd+5zaJ0mSpG5iiNKsKjq1z4YTkiRJ6haGKM06g5QkSZJ6iSFKbTF+H6mp2p+DQUqSJEmdzxCltpqu/TlkQWrV2vVtqEaSJElqnCFKbTU+IlVkep8kSZLUiQxRKsV066T2PmG5XfskSZLUkQxRKs2CgbmukZIkSVLXMUSpVEXXSBmkJEmS1CkMUSpV0TVSBilJkiR1CkOUOoJT+yRJktQtIqVUdg1tFRFDwMjIyAhDQ0Nll6Ma68YeYcmxF0x73vXHH1BoKqAkSZI0mdHRUYaHhwGGU0qjRV/nSJQ6ynRd+8Y5IiVJkqSyGKLUcYoEKVugS5IkqSyGKHWkImukwHVSkiRJaj9DlDrWooXzndonSZKkjmOIUkdzap8kSZI6jSFKHa9oswmn9kmSJKkdDFHqCgYpSZIkdQpDlLqGQUqSJEmdwBClrrJgYC4rTj5w2s59BilJkiTNFkOUutKCgbnTnmOQkiRJ0mwwRKkrObVPkiRJZTFEqWuNT+2bLkwZpCRJktRKhih1vQUDcwutkfJeUpIkSWoFQ5R6QpE1UuColCRJkmYuUkpl19BWETEEjIyMjDA0NFR2OWqxdWOPsOTYCwqde/3xBxQOX5IkSeo9o6OjDA8PAwynlEaLvs6RKPWUIlP7xjkqJUmSpGYYotRzFi2cX6hzHxikJEmS1DhDlHpS0RboYJCSJElSYwxR6lnjLdCLTO8zSEmSJKkoQ5R6np37JEmS1EqGKPW8Rqf2eT8pSZIkTcUQpb4wPrXPdVKSJEmaKUOU+oot0CVJkjRThij1nUZusGuQkiRJUi1DlPpOI2ukwHVSkiRJmsgQpb7USPvzcYYoSZIkgSFKfa6RqX17n7DcICVJkiQipVR2DW0VEUPAyMjICENDQ2WXow6xbuwRlhx7QeHzrz/+gIYCmCRJkjrP6Ogow8PDAMMppdGir3MkSqLx6X2OSEmSJPUvQ5RUpejoklP7JEmS+pchSqrSSOc+u/ZJkiT1J0OUVGN8al8jYWrV2vWzXJUkSZI6hSFKmkQjo1J7n7DcUSlJkqQ+YYiSpuColCRJkmoZoqQCGh2VckRKkiSpdxmipIJsOiFJkiQwREkNaWZ6n0FKkiSptxiipCYsGJjrjXklSZL6VKSUyq6hrSJiCBgZGRlhaGio7HLU5daNPcKSYy8odO71xx9Q+Ga+kiRJmn2jo6MMDw8DDKeURou+zpEoaQZcJyVJktR/DFHSDDUSpMB1UpIkSd3OECW1gEFKkiSpf3REiIqIwyLiTxGxPiKuiogXFHzdmyMiRcS5s12jNB1vzCtJktQfSg9REfEm4HPAicAewKXAjyNip2letzPwmfx8qWM00rnPG/NKkiR1n9K780XEL4GrU0rvrTp2A3BuSmnpJK+ZA/w3cCbwAmDzlNLrCr6f3fnUFo107gO790mSJLVbV3bni4gBYC/gwpqnLgSeO8VLjwXuTSl9tcB7zIuIofEHsFnTBUsNcHqfJElSbyp7Ot8iYA5wd83xu4Ft670gIp4HHAq8u+B7LAVGqh53NFWp1KRGmk7sfcJy26BLkiR1uLJD1LjaOYVR5xgRsRnwNeDdKaVVBa+9DBiueuwwgzqlpti9T5IkqXeUHaJWAY+y8ajT1mw8OgXwJGAxcF5EPBIRjwBvB16T7z+p9gUppQ0ppdHxB7CmpZ9AKmh8el/RphNO75MkSepMpYaolNIYcBWwf81T+wOX13nJjcDTgd2rHj8ALs63b5+1YqUWWbRwfkPT+xyRkiRJ6iyd0J3vTcA5wHuAK4B/JFvv9NSU0sqIOBu4c4pOfWdhdz51Ibv3SZIklasru/MBpJS+CRxO1nHvGuCFwCtTSivzU3YCnlBSedKscZ2UJElSdyp9JKrdHIlSp3FESpIkqRzNjkQZoqQOYZiSJElqr66dzicp00z3Pqf3SZIktZ8hSuowjYwuLTn2Am/OK0mS1GaGKKnDjI9INdJ0whAlSZLUPoYoqUM10r1v7xOWOyIlSZLUJoYoqYM1OirlOilJkqTZZ4iSukAjTSeWHHsBq9aub0NVkiRJ/ckQJXWRRQvnFxqV2vuE5Y5ISZIkzRJDlNRliq6VsnOfJEnS7DBESV2okaYTrpOSJElqLUOU1KUaaTphkJIkSWodQ5TU5ZzeJ0mS1F6GKKkHNDIqZYiSJEmaGUOU1EOKjEp5Y15JkqSZiZRS2TW0VUQMASMjIyMMDQ2VXY40a1atXc/eJyyf9rzrjz+ABQNz21CRJElSZxkdHWV4eBhgOKU0WvR1jkRJPapoMLLphCRJUmMciZL6gKNSkiRJG3MkStKkFi2cbyt0SZKkFjFESX2iaAc/g5QkSdLUDFFSnynSwc97SkmSJE3OECX1IUelJEmSmmeIkvpY0VEpg5QkSVKFIUrqc07vkyRJaowhSpLT+yRJkhpgiJL0F45KSZIkTc8QJWkCR6UkSZKmFimlsmtoq4gYAkZGRkYYGhoquxy450bYbBvYdIuyK5HqWrV2PXufsHzKc64//gAWDMxtU0WSJEmtMTo6yvDwMMBwSmm06OsciSrTue+DLz4bPrUYxh4suxqprkUL5xcalVq1dn2bKpIkSSqXIapMW+1W2d5giFLnGp/id+Ux+016zt4nLHetlCRJ6guGqDI9442V7XuvL68OqaAiU/bGG084MiVJknqVa6LKNPYgnLRdZf/ou2BgsLx6pAasG3uEJcdeMOU5rpWSJEmdzDVRvSA9VnYFUmFFuvjZDl2SJPUiQ1SZBgbho3+s7C/bwQYT6jpFw5RBSpIk9QpDVNnmzi+7AqklFgzMnbLxhKNSkiSpVxiiyjYwCG//QWX/4XXl1SLNkO3QJUlSP7CxRCfYsBaWbV/Zt8GEeoCNJyRJUqezsUQ3i5i4P+ZolLrfgoG5hUalnOInSZK6jSNRnWLlFXDmy7Ptg8+FXV5Sbj1Si61au569T1g+5TmOTEmSpHZyJKrbbblLZftrr7NLn3pO0Rv1ul5KkiR1Ov/Lt1MMLJi4v+5+10Wpp4y3Qoep10uNj1Y5KiVJkjqVI1GdYmAQlt5Z2b/2v8qrRZpl07VDB9dLSZKkzmWI6iTVDSaWfyLr2if1qEUL5097k17wRr2SJKnzGKI6ycAgfPjmyv5tV5RXi9QmdvGTJEndxhDVaeYtrGx//Q02mFBfGF8v5aiUJEnqBoaoTjMwCO/8SWX/wfvKq0VqsyJhylEpSZJUNkNUJ9r2GZXt057uaJT6TtEw9ecRb0wtSZLazxDViQYG4YCTK/sPrS6vFqlE04Wp5yy7mMVHnc+a9Q+3uTJJktTPDFGd6plvqmzfUv9+OlK/mC5MPf24C53iJ0mS2iZSSmXX0FYRMQSMjIyMMDQ0VHY5kxt7EE7arrK/9A6Yt1l59UgdZKqb9YI36pUkScWMjo4yPDwMMJxSGi36OkeiusWKy8quQOoY4yNTK04+kOUffuFGzy859gJ+f5fTYCVJ0uxwJKqT1Y5GHX1Xtl5K0gRTjUz99uP7M7zpQJsrkiRJ3cCRqF40MAjv+GFlf8Oa8mqROthUa6ae+Ymful5KkiS1lCGq0223R2X7s7vZ7lyawniYuu64l2303JJjL2Dlff79kSRJM+fK604XMXF/bJ1T+qRpLJz/OFacfCB/uHcN+332f/5y/EX/cslftm0+IUmSmuVIVKcbGIT3VDWVeGBFaaVI3eZJW22BiFSUAAAgAElEQVQ2aVv0JcdewFrvLyVJkppgiOoGj/+ryvYN55VXh9SFppri97T8/lIPbjBMSZKk4gxR3eby02B94cYhknIL5z9u0lGpp37cMCVJkoozRHWjlT8vuwKpK03VxQ8qYequ1evaXJkkSeom3ieqW3jPKKnlprq/FMDvP/EyBuc9ro0VSZKkdmr2PlGGqG5RG6LAICW10B/vXcNLqzr5VbvuuJexcL5hSpKkXmOIKqhrQxTAhrWwbPvKviFKarkHHtzAHp+8aNLnbY0uSVLvaDZEuSaqm9TeM+qk7bz5rtRiWwzOm3TNFGSt0VevG2tjRZIkqdM4EtVtHloNn9q5su9olDSrVq1dz94nLJ/0eUemJEnqXo5E9Ys5NesyNqwtpw6pTyxaOH/akanFR53PurFH2liVJEkqkyNR3ei+P8Ln96jsOxoltc3qdWPsfvxP6z53zqH78IJdt25zRZIkqVnNjkQ5B6Ubzdus7AqkvrX5ggFWnHxg3fbob/vqr/+y7TQ/SZJ6l9P5utHAgon7NpiQ2m78xr1XHrNf3eeXHHsBF13/5zZXJUmS2sHpfN3Km+9KHefWe9bwN6fUv9eUI1OSJHUe7xNVUM+GKDBISR3ipj+PcsDnLq37nGFKkqTOYYgqqGdCFDgaJXW43981woGn/7zuc4YpSZLKZ4tzuTZK6jBP3W540vboS469gJvvLvxvtSRJ6iCORHW7DWth2faV/SNvhYVblVePpLoe3PAwT/34hZM+78iUJEntZ4vzfhUxcf/RsXLqkDSlwXmPY8XJB04apqrbpRuoJEnqbI5E9YKRO+HUJZV910ZJXeGyW+/lrV/51aTPG6YkSZpdronqZ5tuPnG/z4Kx1K2et8tWk66Zgmx0avFR57Nq7fo2ViVJkqbjSFQvsN251PXWjT0yYUrfVByhkiSpNWxxXlBPhigwSEk9wjAlSVL7GKIK6tkQBfDASjjtGZV9Q5TUtQxTkiTNPrvzCQYXTdzf8KAhSupSCwbmsuLkA4HpA5Wd/SRJai8bS/SSgUF4z+WV/duuKK8WSS0zHqimakIxbrwZxbqxR9pQmSRJ/cnpfL1m7b3wmV0q+07pk3pOI1P9rjxmPxYtnD/LFUmS1J2czqfMwIKJ+w+shG2W1D9XUldqZKrf3icsB5zmJ0lSKzkS1YtqO/U5GiX1NJtQSJLUHLvzFdSXIQoMUlIfWbV2/V9GoKbiVD9JUr8zRBXUFyEKYP0onLxjZd8QJfUdR6gkSZqaIaqgvglRjkZJyjUSpsBAJUnqH4aogvomRAGM3AmnVjWVMERJfc/RKUmSKuzOp41tuvnE/bF1hiipz1V39ptu7VR12HL9lCRJFY5E9bq7r4cv7VvZdzRKUg2n+0mS+pUjUapvcKuyK5DU4apHp2D6UFX9nIFKktSPHInqdTaYkNQkR6gkSb3OxhIFGaJyBilJDbAhhSSpFxmiCuq7EAWORklquXvXPMQ+J/5s2vNsSCFJ6mSGqIL6MkSBQUrSrLl79CGefdL0gQocpZIkdZZmQ9Qms1eSOt7YurIrkNQDthnalOuPP6DQuUuOvYDFR53PurFHZrkqSZJmjyNR/aR2NOrIW2Gh3fsktU6jzSjA0SlJUnmczldQX4cogJsvgP94Y2XfKX2SZomBSpLU6bxPlIp5wh5lVyCpT1Tff6pooPIeVJKkbuBIVL+xwYSkDuA9qCRJncCRKElS12h0lGr8ecOUJKkTOBLVb7z5rqQOtmrtevY+YXnh8w1VkqSZsLFEQX0foqB+kLJTn6QOY6CSJM02p/NpZh57tOwKJGmCRQvnNzXlDwxUkqTZ5UhUv7LBhKQu1Ezb9CuP2Y9FC+fPUkWSpG7mdL6CDFFV1t4Ln9mlsr/0Tpi3sLx6JKkBBipJ0kwZogoyRFVxNEpSj2h0/dQ4Q5Uk9TdDVEGGqCp26pPUg5oZoQLXUUlSPzJEFWSIquFolKQe1mygAkOVJPUDQ1RBhqg6DFKSetxMwtQ4Q5Uk9R5bnEuSNIkFA3MbapdeT/VrXEslSf3NkShlakejvPmupD7RbFOKWo5USVL3cTpfQYaoKVz/A/jW27Ltw34JWz+53Hokqc1aFajAUCVJ3cAQVZAhagpr7oHP7lrZd12UpD7WinVU1QxVktR5XBOlmZtXE5g2rDFESepbrVhHVa369QYqSepujkSpwvtGSdK0Wj1CBYYqSSqL0/kKMkRNw3bnktQwg5UkdSdDVEGGqAJqg9Q/XQ/D25dXjyR1idkIU9UMVpLUWoaoggxRBd3/Jzh992x7v4/DC44otx5J6jKzHajGGawkqXmGqIIMUQWtvRc+s0tl/6jbYP5wefVIUo9oR7jyZsCSVIwhqiBDVEG1U/pedwbsflB59UhSj2rXiNU4R64kqcIQVZAhqgG1QWrpnTBvYXn1SFIfaOUNf4tw1EpSPzNEFWSIatADt8FpT8+23/od2PVvyq1HkvpMu0eqwNEqSf3DEFWQIapBtaNRtjuXpNKVEazAcCWp9xiiCjJENeHP18IZz8+2330JbL9HqeVIkiYqK1Q5FVBStzNEFWSIatI3DoKbfpRtOxolSR2trFAFjlZJ6i6GqIIMUU36wyVwzmuz7dd9CXZ/S6nlSJIa0+6GFbUMV5I6kSGqIENUk2rXRh1xIww9obx6JEkzVuaIFRisJJXPEFWQIWoGHlwF//KkbPsFH4b9ji23HklSy5UdrMBwJal9DFEFGaJm6JpvwLnvybb/6fcwvEO59UiSZl0nBCswXElqPUNUQYaoGdqwBpZVBSebTEhS3yp7nVUtQ5akRhmiCjJEtcC134bvHJptf+RWGNyq3HokSR2jU0atqhmuJE3GEFWQIaoFHnsUTt8dVt+W7TsaJUmaRqeNWlUzZEn9yxBVkCGqRf7703Dxidn2+34NW/11ufVIkrpOJ45a1TJgSb3NEFWQIapFHnoAPrU42973/XDAiaWWI0nqHd0QrsZdecx+LFo4v+wyJDXJEFWQIaqFrvsOfPud2fYHroItdym3HklST+umcFWPgUvqPIaoggxRLWSnPklSybo9WE3FqYTS7DNEFWSIarE/XwtnPD/bftu58KSXlFuPJEn0drgqwgAmFWOIKsgQNQvOfS9c8x/ZtqNRkqQO1+8Ba5zTCyVDVGGGqFlw+6/hq3+Tbb/hLHja60stR5KkVjFwzYwjYup0hqiCDFGzYOxBOGm7yv5Rt8N8v7aSpP7QyffA6maOlKkdDFEFGaJmyerb4XNPy7YP/Czs865y65EkqQM5stV+joZpKoaoggxRs6R2NOojt8LgVuXVI0lSjzB4dQ4DWe/p6hAVEYcBHwGeAPweODyldOkk5/4tcDSwC/A44Bbgsymlcwq+lyFqtqy7Hz79V5V9m0xIktRWBq7+5fTH5nRtiIqINwHnAIcBlwH/H/AuYElK6bY6578Y2AK4ERgDXgV8FjgwpTTtvxqGqFl21Vlw3oey7SNugKHtpjxdkiSVx/VcKlMnjOx1c4j6JXB1Sum9VcduAM5NKS0teI2rgfNTSv9c57l5wLyqQ5sBdxiiZsljj8IX94VVN2X7jkZJktTzDGNqRjeHqFKrjogBYC/g5JqnLgSeW+D1AbwU2A342CSnLQU+PoMy1YhN5sDzPgTfPyzbf+B22ObJ5dYkSZJm1aKF81lx8oGTPu80Q9Wz4eFHSw9RzSq76kXAHODumuN3A9tO9qKIGAbuJBthehQ4LKX000lOXwacUrW/GXBHswWrgCWvrYSoX3wBXvuFcuuRJEmlWjAwd8qQNR1DWG9a/dDDbDE4b/oTO1DZIWpc7ZzCqHOs2hpgd2AhsB9wSkT8MaV0yUYXTmkDsOEvF46YcbGaxryFcPD34Guvh9+cA7u/BXaedmBRkiSprpmGsGrVgezKY/ZzGmKJNpvfKVGkcWVXvopsJKl21GlrNh6d+ouU0mPArfnuNRHxFLJpe5fMQo1qxk7Prmyf+QpYegfM26y8eiRJktg4kLUqnDWrn0fZunUqH5QcolJKYxFxFbA/8L2qp/YHvt/ApYKJzSNUtoFB+MDV8Pk9s/1lO9hkQpIkqUYrR9mq2exjdnVC/DsFOCcirgSuAP4R2Ak4AyAizgbuHO/UFxFLgSuBPwADwCuBtwPv3fjSKtWWT4IXL4VLlmX7Y+sMUZIkSW0wXbMPzcwmZReQUvomcDhwLHAN8ELglSmllfkpO5HdhHfcIPBFspvyXg68ATg4pfSVthWt4nY/uLJ9+enl1SFJkiS1SOn3iWo3b7bbZmMPwklVN9x9/5WwaNfy6pEkSZJyzd4nqvSRKPW4gUFYemdl/5LaW4JJkiRJ3cUQpdk3byH8w4+z7eu+DSsuK7ceSZIkaQYMUWqPJzyzsn3WK2HDmvJqkSRJkmbAEKX2GG95Pm7ZDtl6KUmSJKnLGKLUPls+CV56bGX//j+VV4skSZLUJEOU2utZ765s/8+/lFeHJEmS1CRDlNprkzmV7evPhVt/Vl4tkiRJUhMMUWqvgUE4+q7K/tdeD+sLt+SXJEmSSmeIUvsNDMKHrq3sn7yjTSYkSZLUNQxRKscWO8HLTqjsn7SdQUqSJEldwRCl8uz59rIrkCRJkhpmiFJ55g/DG79W2b/jqvJqkSRJkgoyRKlcOz2nsn32q2HD2vJqkSRJkgowRKlcAwsm7i/b3rVRkiRJ6miGKJWrtuU5wIP3lVOLJEmSVIAhSuUbGISPrazsn/Z0R6MkSZLUsQxR6gybbg5v/W5l/5aflleLJEmSNAVDlDrHzlVNJv7n0/DYY+XVIkmSJE3CEKXOMTAIh1+Xbd/9e/jdf5ZbjyRJklSHIUqdZcHjK9vnvhceWl1eLZIkSVIdhih1loFBOPKWyv5vHY2SJElSZ5lxiIqIoYh4XUQ8pRUFSSzcGvb/ZLb9k4/BfbeWW48kSZJUpeEQFRHfioj359ubAlcC3wJ+FxF/1+L61K/2fFtl+/N7wfqR8mqRJEmSqjQzEvVC4NJ8+/VAAJsDHwSOaVFd6nebbgGHXlTZv/xfy6tFkiRJqtJMiBoG7s+3Xw58J6W0Djgf2LVVhUlss6Sy/T+fclqfJEmSOkIzIep2YN+IGCQLURfmx7cA1reqMImBQVh6R2X/83vB2IPl1SNJkiTRXIj6HPB14A7gLuCS/PgLgWtbU5aUm7cZHHphZf+k7QxSkiRJKlXDISql9EVgX+CdwPNTSo/lT/0R10RpNmzztIn7BilJkiSVqKkW5ymlK1NK30sprY2IORGxO3B5SumyFtcnZdP6/un6squQJEmSgOZanH8uIg7Nt+cA/w1cDdweES9ubXlSbnh7OPDUyv6ff19eLZIkSeprzYxEvQH4bb79auCvgCeTrZU6sUV1SRt7xhsr2/+2P2xYW14tkiRJ6lvNhKhFwJ/z7VcC/5VSuhn4KvD0VhUmbWTeQnjvFZX9675TXi2SJEnqW82EqLuBJflUvpcD43dEXQA82qrCpLq22Lmyfd4HYfVt5dUiSZKkvtRMiDoT+BZwHZCAn+bHnw3c2KK6pPoGBuFjKyv7n3u6nfokSZLUVs20OD8OeBfwZeB5KaUN+VOPAie3rjRpEptuDm/5VmX/1p+VV4skSZL6TqSUyq6hrSJiCBgZGRlhaGio7HLUrLEHs/tFjfvon2DB48urR5IkSV1ndHSU4eFhgOGU0mjR1zV1n6iIeFFEnBcRt0bELRHxg4h4QTPXkpoyMAhH3FDZ//RfOa1PkiRJbdHMfaIOJmsmsQ44HfgC8BCwPCLe0trypCnMH564f9+t5dQhSZKkvtLwdL6IuAH4ckrp1JrjRwDvTik9pYX1tZzT+XrMhrWwbPvK/tI7s1bokiRJ0jTaOZ3vicB5dY7/gOzGu1L7zFsIh/2ysr9se6f1SZIkaVY1E6JuB/arc3y//DmpvbZ+MrzwY5X91XeUV4skSZJ6XjMh6rPA6RHxpYh4W0QcHBFnAKcBn2lteVJB+x5W2f7isxyNkiRJ0qyZ2+gLUkpfiog/Ax8G3pgfvgF4U0rp+60sTipszuMm7q/4Ofz1AeXUIkmSpJ7WsvtERcTjgCeklG5ryQVniY0lethG9476IyzYsrx6JEmS1NHaep+oSSwB/tTC60mNGRiEI26s7H/6iU7rkyRJUsu1MkRJ5ZtfM7r4v78rpw5JkiT1LEOUesvAIHz4lsr+T/8ZHnusvHokSZLUcwxR6j3zBivbd/warvxqebVIkiSp5xTuzhcRz5jmlN1mWIvUGgODcPRdlSYTPzoSdn0ZbLFzuXVJkiSpJxTuzhcRjwEJiDpPjx9PKaU5rSuv9ezO10fWj8DJO1X2l94J8xaWV48kSZI6Sju68/0V8MT8z9rHE6v+lDrD/GE45EeV/WXb261PkiRJM1Z4Ol9KaeVsFiLNiu12n7j/wErYZkk5tUiSJKkn2FhCva22W9+X9oUNa8urR5IkSV3PEKXeV92tD5zWJ0mSpBkxRKn3jXfrq3bSdgYpSZIkNcUQpf4wMJh156tmkJIkSVITDFHqH/MWwrsvnnhsbF05tUiSJKlrFe7ONy4ifkN2X6haCVgP3AqclVK6uM45Urm2qrkn9Gd2yab6DQzWP1+SJEmq0cxI1E/I7gf1IHAxcAmwFngS8GvgCcBFEfHaFtUotc7AIPzTDROPOa1PkiRJDWh4JApYBHw2pfTJ6oMRcQywc0rpZRHxCeCfge+3oEaptTYdLrsCSZIkdbFmRqLeCHyjzvH/zJ8jf363OudIncnRKEmSJBXUTIhaDzy3zvHn5s+NX3dDs0VJs6pey3OwyYQkSZIKaSZEfR44IyJOi4iDI+KtEXEa8CXg9PycA4DftKpIqeXqBanP7OJolCRJkqYVKdVrtDfNiyLeCryfypS9m4DPp5T+I39+UyCllNZPconSRMQQMDIyMsLQ0FDZ5ahsa+6Bz+468Zjd+iRJkvrC6Ogow8PDAMMppdGir2umsQQppa8DX5/i+Yeaua7UdhEbHztpO4OUJEmSJtX0zXYjYq+q6Xx7tLIoqW0WblV/fZSNJiRJkjSJZm62uzVZJ74XA6uBAIYj4mLgzSmle1taoVSWsXWORkmSJGkjzTaWGAKemlJ6fEppC+Bp+bHTp3yl1Ikm69ZnowlJkiTV0UyIejnw3pTSDeMHUkrXA+8DXtGqwqS2mixIOa1PkiRJNZoJUZsAD9c5/nCT15M6w8AgHHnrxscNUpIkSarSTOj5GXBaRGw3fiAitgdOBZa3qjCpFAMLyq5AkiRJHa6ZEPV+YDNgRUT8ISJuBf6UH/tgK4uT2s5pfZIkSZpGw935Ukq3A3tGxP7Ak8m6812fUrqo1cVJHcVufZIkSQIipdSaC0XsCHwipfTOllxwlkTEEDAyMjLC0NBQ2eWok409mI1AVfMmvJIkST1jdHSU4eFhgOGU0mjR17WyEcTjgXe08HpSuQYG4YgbJx5zWp8kSVLfs5ueNJVNGp7xKkmSpB5niJKmUq9bn6NRkiRJfc0QJU1lsm596x5ofy2SJEnqCIXnKkXEd6c5ZfMZ1iJ1pvEgVd1k4nNPtcmEJElSn2pkJGpkmsdK4OxWFyh1hIFBOPLWicec1idJktSXCo9EpZT+YTYLkTreZOujHJGSJEnqK66JkoqabH2UI1KSJEl9xRAlNWKyIDW2rv21SJIkqRSGKKlR9YLUXb8ppxZJkiS1nSFKasbAIHz4lsr+f/w9jNxRXj2SJElqG0OU1Kx5Nc0kTn0qbFhbTi2SJElqG0OU1Kx60/qWbW+TCUmSpB5niJJmol6Qemh1ObVIkiSpLQxR0kwNDMIHqhpLnLrE0ShJkqQeZoiSWmGzbSbue+8oSZKknmWIkmaLQUqSJKknGaKkVpjsJrwGKUmSpJ5jiJJaZbIgNbau/bVIkiRp1hiipFaqF6Q+s4ujUZIkST3EECW12sAgHHnrxGNO65MkSeoZhihpNgws2PiYQUqSJKknGKKk2WCjCUmSpJ5liJJmi40mJEmSepIhSppNNpqQJEnqOYYoabYNDMIHfzvxmNP6JEmSupYhSmqHhVttfMwgJUmS1JUMUVI7uD5KkiSpZxiipDK5PkqSJKnrGKKkdrHtuSRJUk8wREntNDAIR9668XGDlCRJUtcwREntNrCg/nGDlCRJUlcwREntNtm0PrDRhCRJUhcwREllmCxI2WhCkiSp4xmipLLYaEKSJKkrGaKkMtloQpIkqesYoqSy2WhCkiSpqxiipLLZaEKSJKmrGKKkTmCjCUmSpK5hiJI6xVQjUpIkSeoYhiipk9RrNOHaKEmSpI5iiJI6Tb1GEwYpSZKkjmGIkjrNZNP6Vt/R/lokSZK0EUOU1IkGBuG4EXj/1ZVjX3wWrB8tryZJkiQBhiipsw1tO3H/5B2d1idJklQyQ5TUyepN7XN9lCRJUqkMUVKnM0hJkiR1FEOU1A0GBuGjf5x47N6by6lFkiSpzxmipG6xYEv4wG8q+//3xbBhTWnlSJIk9StDlNRNNttm4v6yHZzWJ0mS1GaGKKmbuD5KkiSpdIYoqdvUC1J3XVNOLZIkSX3IECV1o4FBOOKmyv5Zr4S195RXjyRJUh8xREndav5mE/d/9BFIqZxaJEmS+oghSupWtdP6rj8XfvGl8uqRJEnqE4YoqZvVBqkLlsLKy8urR5IkqQ8YoqRuNzAIS++s7J/5Chi5c/LzJUmSNCOGKKkXzFsI77+6sn/qElh3f3n1SJIk9TBDlNQrFj0J3nVRZX/58eXVIkmS1MMMUVIv2WEfeOM52fZVZ8Jxw96IV5IkqcUMUVKvWfIa2Pd9lf2TtjNISZIktZAhSupFf3M87Py8yv4DK8urRZIkqccYoqReNGcuvPr0yv6X9oV195VXjyRJUg8xREm9atEu8J6fV/Y//UTYsLa8eiRJknqEIUrqZds+Hd7yrcr+su1dHyVJkjRDhiip1/31AbD/Jyv7N/64vFokSZJ6gCFK6gf7HFrZ/u6hsPLy8mqRJEnqch0RoiLisIj4U0Ssj4irIuIFU5z77oi4NCIeyB8XRcSz2lmv1HUGBuFjKyr7Z74C7rmxtHIkSZK6WekhKiLeBHwOOBHYA7gU+HFE7DTJS14MfAN4CbAvcBtwYURsP/vVSl1s0y3gwzdX9r/4bFhzd3n1SJIkdalIKZVbQMQvgatTSu+tOnYDcG5KaWmB188BHgDen1I6u8D5Q8DIyMgIQ0NDM6hc6lKrboEv7F3ZX3oHzNusvHokSZJKMjo6yvDwMMBwSmm06OtKHYmKiAFgL+DCmqcuBJ5b8DILgMcB90/yHvMiYmj8Afjbovrbol3hrd+u7J//YSj5P1MkSZK6SdnT+RYBc4DaOUV3A9sWvMbJwJ3ARZM8vxQYqXrc0XiZUo/Zuer/KH73TfjZJyc/V5IkSROUHaLG1f43eNQ5tpGI+ChwEPC3KaX1k5y2DBiueuwwgzql3jAwCMeNwCv+Jdu/9LNw9Tnl1iRJktQlyg5Rq4BH2XjUaWs2Hp2aICKOBI4GXpZS+t1k56WUNqSURscfwJoZ1iz1jmf/IzznsGz7B++HW39Wbj2SJEldoNQQlVIaA64C9q95an9g0hvZRMRHgH8GXp5SunL2KpT6wMtOhN1emW1/7fVw+6/KrUeSJKnDlT0SBXAK8K6IeGdEPCUiTgV2As4AiIizI2LZ+Mn5FL4TgHcCKyJi2/yxsIzipa63ySbwmtMr+1/dH1bdPPn5kiRJfa70EJVS+iZwOHAscA3wQuCVKaWV+Sk7AU+oeslhwADwbeB/qx5HtqtmqecMbgVH3FDZ/8I+sO6+8uqRJEnqYKXfJ6rdvE+UNIV7boAvPifbfvKr4e/PgjlzSy1JkiRptnTlfaIkdZitnwIHfTPbvvE8uODocuuRJEnqQIYoSRPt9nJ4/Zez7V/9H7j67HLrkSRJ6jCGKEkbe8qrKts/+AD8wdbnkiRJ4wxRkjY2MAhL76jsn/N6WHFZefVIkiR1EEOUpPrmbQZH3lLZP+uV3oxXkiQJQ5SkqSzcGj5cdc+or70e7r6+vHokSZI6gCFK0tQ222ZikPrSvrD69vLqkSTp/7V359GWXXWdwL8/EiuBqqQYUkAqIYCGMEPoMLcgimBr93IxtMpq7AX8obQspkAYgoKxsQM0CQQSW1vtNg2KiHOD2Ew2M7iYZEYooJDMAUINCamCcPqPe27n1qs3nPvene/ns9ZZ790z3X2Tnbz3fXvv34EpE6KAjZ1wh+TZn7r59UX38TBeAGBpCVFAN7f90eSpb7/59V89Pbnp+9NrDwDAlAhRQHe7z7z5+z3vSl5+UnJj54d7AwAsBCEK6G7b9uQlVxy575V3Sg4dmE57AACmQIgChrNakHrFqcn+K6fTHgCACROigOGtFqRecw9T+wCApSBEAZuz5tS+g9NpDwDAhAhRwOatOrXvlOTw9dNpDwDABAhRwNasFqTO3y1IAQALS4gCtm61IPWZP59OWwAAxkyIAkZj2/bk3Mtufv225yTn7TQiBQAsHCEKGJ3jTkhe/M0j95naBwAsGCEKGK3jT0xe/C9H7tv7oem0BQBgDIQoYPSO35mc/YWbX7/pF5LLPzW99gAAjJAQBYzHzlOSc/bc/PoPHmWNFACwEIQoYHy23erofdZIAQBzTogCxmfb9uS8fckzP3bk/gNXT6c9AAAjIEQB43fSGcnT3nHz64sfYGofADC3hChgMk6+79H7TO0DAOaQEAVMxrbtyUuuOHq/IAUAzBkhCpgcQQoAWABCFDBZ/WITT/6rI/cLUgDAnBCigOm480OP3idIAQBzQIgCpsPUPgBgTglRwPQIUgDAHBKigOnatj05Z8/R+8/f7VlSAMBMEqKA6duxa/URqSQ5fMNk2wIAsAEhCpgNa03tu+B0I1IAwEwRooDZ0S9//rx/PvqYdVIAwIwQooDZc/wJq2qz88QAABqnSURBVO8XpACAGSBEAbNnral9iSAFAEydEAXMpv7UvrUq9wlSAMCUCFHAbNt2q9X3n787OXjtZNsCABAhCph1603tu+B0I1IAwMQJUcDs22iNlBLoAMAECVHAfFgvSCXWSQEAEyNEAfNjoyB1+IbJtQUAWFpCFDBfNlojZWofADBmQhQwf0ztAwCmSIgC5tN6z5FKTO0DAMZGiALm21rPkVL+HAAYEyEKmG/KnwMAEyZEAfNvo6l91kgBACMkRAGLY62pfYk1UgDAyFTTNNNuw0RV1YlJ9u3bty8nnnjitJsDjMPh63ujT6t5yRW9kSsAYOnt378/O3fuTJKdTdPs73qdkShg8Wy0TsrUPgBgC4QoYDFt277+GqmD1062PQDAwhCigMW1Y9faI1JKoAMAmyREAYtNCXQAYMSEKGDxbVQCXeU+AGAIQhSwPNYqgW5qHwAwBCXOgeWjBDoAECXOAbpTAh0A2AIhClhOghQAsElCFLC8+gUnVgtTghQAsAYhCmCtB/N6KC8AsAohCiBZ+8G8F5zuWVIAwBGEKIA+66QAgA6EKIBBa03tS0zvAwCSCFEAR1vrobyJB/MCAB62C7Cm9R7Km3gwLwDMOQ/bBRi1fgn09ab3GZUCgKUjRAFsZL3pfYIUACwdIQpgI+s9lDcRpABgyQhRAF1tVAJd5T4AWApCFMAw1gtSHswLAEtBdT6AzVK9DwDmmup8AJO23oN5k+TwDZNrCwAwMUIUwFbs2GV6HwAsGSEKYKtU7wOApSJEAYzKetP7VO8DgIUhRAGM0noP5r3gdCNSALAAVOcDGAeV+wBg5qnOBzBL+uuk1prep3IfAMwtIQpgnNaa3qdyHwDMLdP5ACbB9D4AmDmm8wHMMg/mBYCFIUQBTIoH8wLAQhCiACbJg3kBYO4JUQDTsNGDeQUpAJhZQhTAtKw3ve/83ab3AcCMEqIApmmj50kZlQKAmSNEAcyCtZ4nlQhSADBjPCcKYJZ4nhQATIznRAEsgm3b114nlfQC1sFrJ9ceAOAoQhTArNkoSHmmFABMlRAFMIs2ep5UYlQKAKbEmiiAeXDw2t4I1FqslQKAoVkTBbDI1qvel6jgBwATJEQBzAPT+wBgZghRAPNE0QkAmDprogDm1UbPlEqslQKAdVgTBbBs+lP8ztmz9jnWSgHAyAlRAPNux66N10oJUgAwMkIUwCLYqPCEohMAMDJCFMAiWa/whKITADASCksALKqNCk+cs6c3FRAAlpTCEgAcadv29YtOGJkCgE0RogAW2UZFJxLrpQBgSKbzASwLz5UCgCOYzgfA+tYrOtF3/m5T/ABgA0IUwDLZqBR6n2dLAcCaTOcDWFZdpvf1meYHwAIynQ+A4XQdlUqSwzeMvz0AMCeMRAFgVAqApWQkCoDN61J0os96KQCWnJEoAI5kVAqAJWEkCoDR6K+VOmfPxud6UC8AS0iIAmB1O3Z1m+J3wemeLQXAUhGiAFibUSkAOIoQBcDGhhmVMiIFwIJTWAKA4XQtPKHoBAAzTmEJACajazl00/sAWFBGogDYmoPX9qbxrceoFAAzyEgUANPRZb3U+bt7FfyMTAGwAIQoALau6xQ/5dABWABCFACjoRw6AEvCmigARq9rBb8+a6YAmAJrogCYHf1RqS5T/BIjUwDMFSNRAIzfMCNTRqUAmJDNjkQJUQBMTpdy6H3CFABjZjofALOvSzn0vvN3q+IHwEwyEgXA9HQdmTpnTy+AAcAIGYkCYP50HZm64HSjUgDMDCNRAMwGxScAmDAjUQDMt23brZcCYC4YiQJgNnVdL2VUCoBNMhIFwGLpul7KqBQAE2YkCoD50GVkyqgUAEMwEgXAYusyMnX+7uS8nUamABgrI1EAzB/rpQAYASNRACyPHbuS8/Z1G5k6eO1k2gTA0jASBcB883wpADbJSBQAy2nb9m6jUolKfgCMhJEoABZL15Epo1IAS29uR6Kq6hlV9fWqurGqPlFVj1jn3HtX1V9W1d6qaqrquZNsKwBzoOvIlEp+AGzSVENUVf1SkouS/JckD0jygSR/X1WnrXHJrZJ8LcmLk1w1kUYCMJ+EKQDGZKrT+arqH5N8smmaXxvY98Ukf9M0zbkbXLs3yUVN01w05HuazgewjEzzA2CFuZvOV1XbkpyV5J0rDr0zycNH+D7HVdWJ/S3JCaO6NwBzpD8ydc6e9c8zMgXABqY5ne+kJMckuXrF/quT3HGE73Nukn0D22UjvDcA82aYZ0wJUwCsYuqFJZKsnE9Yq+zbilck2TmwnTrCewMwr4xMAbBJx07xvb+V5KYcPep0+xw9OrVpTdMcSnKo/7qqRnVrABZBf2RqozVT/WPWTAEsvamNRDVNczjJJ5I8ZsWhxyT58ORbBMBSU80PgI6mORKVJK9J8saq+niSjyT51SSnJfm9JKmqNyS5vF+pry1Gca/22m1JTqmqM5McbJpmg/kYANBBP0x1HZlKjE4BLJmpljhPeg/bTfLCJCcn+VySs5umeX977L1J9jZN89T29V2SfH2V27yvaZpHdXw/Jc4B6K5rafREmAKYM5stcT71EDVpQhQAmyJMASwcIaojIQqALfPgXoCFMHcP2wWAuTVMEYoD10ymTQBMjJEoABiFg9cmF5y+/jlGpgBmiul8HQlRAIxVl6l+whTATBCiOhKiAJgI66YAZp4Q1ZEQBcBECVMAM0thCQCYRcMUodh/1WTaBMCWGIkCgElThAJgJpjO15EQBcDMUIQCYKqEqI6EKABmTpcwdfYXk50d1lYB0JkQ1ZEQBcBMO3BNcuHd1j/H6BTASAhRHQlRAMwFU/0Axk6I6kiIAmCuKJEOMDZCVEdCFABzS1U/gJESojoSogCYe0anAEZCiOpIiAJgYXQNU+fsSXbsGn97AOaMENWREAXAQuoy1a/PCBVAEiGqMyEKgIXWdXQqEaaApSdEdSREAbAUhCmADQlRHQlRACwdgQpgVUJUR0IUAEtLmAI4ghDVkRAFABGoACJEdSZEAcAKKvsBS0qI6kiIAoB1HLgmufBu3c4VqIA5J0R1JEQBQAem+wFLQIjqSIgCgCEJVMCCEqI6EqIAYAusnwIWiBDVkRAFACMwzOhUIlABM0mI6kiIAoARE6iAOSVEdSREAcCYDBumEoEKmCohqiMhCgAmZJj1U4lABUycENWREAUAE2aECphRQlRHQhQATJFABcwQIaojIQoAZsBmwlSSnLMn2bFr9O0BlpIQ1ZEQBQAzZrOByggVsEVCVEdCFADMMIEKmCAhqiMhCgDmiDVUwBgJUR0JUQAwpwQqYMSEqI6EKABYAKb9ASMgRHUkRAHAghGogE0SojoSogBggQlUwBCEqI6EKABYEp5FBWxAiOpIiAKAJXXw2uSC04e/zigVLCwhqiMhCgDYdKAySgULRYjqSIgCAI5ilAqWkhDVkRAFAKxLcQpYGkJUR0IUANCZQAULTYjqSIgCADZNqIKFIkR1JEQBACMhUMHcE6I6EqIAgLHZTIEKoQqmRojqSIgCACZiMyNVL/pGcstbj6c9wFGEqI6EKABgKjYzSnXuZclxJ4ynPYAQ1ZUQBQDMhAPXJBferfv5z/jH5Pb3GF97YAkJUR0JUQDATDp0IHnFqd3Pf+bHk5OGCGHAUYSojoQoAGAufO+65FV3Ge4aRSpgKEJUR0IUADCXDlyVXHj34a4RqmBdQlRHQhQAsBD2X5m8Zog1Uk//QHLy/cbXHphDQlRHQhQAsHA2U079yX+R3PWRybHHjadNMAeEqI6EKABgKVz3jeR1Q448PeOjya57JFXjaRPMGCGqIyEKAFhKB65OLjxjc9daW8WCEqI6EqIAAJI0TXL5J5M//Knhr33xN5Pj/R7F/BOiOhKiAABWsZl1VYOMVjGHhKiOhCgAgI4EKxacENWREAUAsElbDVXn7El27Bpde2CLhKiOhCgAgBH7zt7k9fff3LXP+1Jy4skjbQ50JUR1JEQBAEzAvsuS1957c9ee/flk56mjbQ+sQojqSIgCAJiSb30lueSBm7v22Z9KbnNXz7BipISojoQoAIAZsdU1Voly62yJENWREAUAMOP2XZ689l6bv946KzoSojoSogAA5tB1/5K87r5bu4dRK1YQojoSogAAFsShA8krtlCA4j/+bXLaQ5MfOX50bWKuCFEdCVEAAAvuW3uSS87a2j2MWi0FIaojIQoAYAltddSq79zLk+N2bP0+zAQhqiMhCgCAJEnTJN/5WnLxv9rafZ71qeR2PzqaNjFRQlRHQhQAAOu6cX/yyjtt/T4v/Fpyq9tt/T6MjRDVkRAFAMCm7Lsiee09t36fZ/9Tcpu7eHDwDBCiOhKiAAAYqe9+M7noPlu/z9PekZzygOTY47Z+LzoRojoSogAAGLtDB5NXnDLae579xWTn7tHec8kJUR0JUQAATNV130xeN4KRq0HP+Wxym9NGe88lIER1JEQBADCTmia55kvJ7z50tPd9wVeT7SeN9p4LQojqSIgCAGDujOo5Vyu9aG9yy9uM/r5zQojqSIgCAGDh7Lssee29R3e/Z3w0OemM5BbHjO6eM0iI6kiIAgBgaTRNr3rg6+472vs+5zPJrU+b+zLtQlRHQhQAAGQ8FQT7XrAn2b5rPPceISGqIyEKAAA28MObkm/vSX7nwaO/91Penuy+X3LcCaO/95CEqI6EKAAA2KIfHE6+/ZXkdx+++Xs8/QPJyfcbXZs2YbMh6tjxNQkAAFhIx25L7nDv5Lx9qx//3neTKz+dvOHn177HifP74GAjUQAAwGQdOjDX0/luMb4mAQAArGIGAtRWCFEAAABDEKIAAACGIEQBAAAMQYgCAAAYghAFAAAwBCEKAABgCEIUAADAEIQoAACAIQhRAAAAQxCiAAAAhiBEAQAADEGIAgAAGIIQBQAAMAQhCgAAYAhCFAAAwBCEKAAAgCEIUQAAAEMQogAAAIYgRAEAAAxBiAIAABiCEAUAADAEIQoAAGAIQhQAAMAQhCgAAIAhCFEAAABDEKIAAACGIEQBAAAMQYgCAAAYghAFAAAwBCEKAABgCEIUAADAEIQoAACAIRw77QZMy/79+6fdBAAAYIo2mwmqaZoRN2W2VdUpSS6bdjsAAICZcWrTNJd3PXkZQ1Ql2Z3kwLTbwpackF4YPjX+XTIZ+hyTpL8xafockzZLfe6EJFc0QwSjpZvO1/7D6ZwymU29LJwkOdA0jbmZjJ0+xyTpb0yaPsekzVifG/r9FZYAAAAYghAFAAAwBCGKeXUoyW+1X2ES9DkmSX9j0vQ5Jm2u+9zSFZYAAADYCiNRAAAAQxCiAAAAhiBEAQAADEGIAgAAGIIQxcyoqnOr6mNVdaCqrqmqv6mqu68457iquriqvlVV11fV/66qU1ecc1pVvbU9/q2qen1VbZvsp2HetP2vqaqLBvbpb4xUVZ1SVX9cVd+uqhuq6p+q6qyB41VV51XVFVX1vap6b1Xde8U9blNVb6yqfe32xqq69eQ/DbOuqo6tqt+uqq+3/elrVfWyqrrFwDn6HJtWVY9sfwZe0f4MfdyK4yPpX1V136p6X3uPy9t+XJkiIYpZ8hNJfifJQ5M8JsmxSd5ZVdsHzrkoyeOTPCnJjyfZkeRtVXVMkrRf/y7J9vb4k5I8McmFE/oMzKGqelCSX03ymRWH9DdGpqpuk+RDSb6f5GeT3CvJ85N8d+C0FyZ5XpJnJnlQkquSvKuqThg4501Jzkzyb9rtzCRvHHf7mUsvSvKf0utP90yvf70gybMGztHn2IrtST6dXv9ZzZb7V1WdmORdSa5o7/GsJOe0952epmlstpnckuxK0iR5ZPt6Z5LDSX5p4JzdSW5K8jPt659tX+8eOOdJSW5McuK0P5Nt9rb0gtGXk/x0kvcmuajdr7/ZRroleWWSD6xzvJJcmeRFA/uOSy9kPb19fc/2/4sPGTjnoe2+u0/7M9pma0vytiT/Y8W+v0zyxvZ7fc42sq3tE48beD2S/pXk19prjhs458VJLk/7uKZpbEaimGU726/fab+eleRHkryzf0LTNFck+VySh7e7Hpbkc+3+vnek9x/tWYGj/U6Sv2ua5t0r9utvjNrPJ/l4Vf15O2X5U1X1KwPH75rkjjmyzx1K8r4c2ef2NU3zjwPnfDTJvoFzoO+DSR5dVWckSVXdP71R87e3x/U5xmlU/ethSd7XXtv3jvT+sHmXcTV+I8dO641hPe0819ck+WDTNJ9rd98xyeGmaa5bcfrV7bH+OVcPHmya5rqqOjxwDiRJqupJ6YWdB65yWH9j1H40vb+ovibJ+UkenOT1VXWoaZo35OY+c/WK665Ocuf2+zsmuWaVe18TfY6jvSq9P0h+qapuSnJMkl9vmuZP2+P6HOM0qv51xyR7V7lH/9jXt9TKTRKimFWXJLlfen8x20ilN+zb13Q4hyVXVXdK8rokj22a5sZhLo3+xubcIsnHm6Z5Sfv6U+0C619L8oaB81b2HX2OzfqlJL+c5D8k+Xx6a00uqqormqb5XwPn6XOM0yj612r3WOvaiTCdj5lTVRenN+3lJ5umuWzg0FVJtrWLswfdPjf/ReKqrPjLWHv+j+Tov4Sw3M5Kr+98oqp+UFU/SK+4ybPb76+O/sZoXZnkCyv2fTHJae33V7VfV/51f2Wfu8Mq994VfY6jvTrJK5umeXPTNJ9tmuaNSV6b5Nz2uD7HOI2qfx31s7a9RzLFPihEMTPaMpiXJHlCkp9qmmbl8Own0qtq9ZiBa05Ocp8kH253fSTJfdr9fY9Ncqi9Hvrek+S+6f1ltr99PMmfDHyvvzFKH0py9xX7zkjyjfb7r6f3y8Jgn9uWXrgf7HM7q+rBA+c8JL0pW/1zoO9WSX64Yt9Nufn3P32OcRpV//pIkkeueHzIY9Or1rd3XI3f0LQredhs/S3Jf0uv+spPpPcXh/52y4FzfjfJN5M8OskD0vtF+J+SHNMePybJZ5O8uz3+6Pb8i6f9+Wyzv2WgOl/7Wn+zjWxLrzTv95O8JMnp6U2xuj7JkwfOeVH7/8HHpxfY35TeLwonDJzz9+mVFH5ou30myVun/flss7cluTTJZUn+bXoL8B+f5Nokrxo4R5+zbXpLr8Jt/w+RTZKz2+9Pa49vuX+lF6iuaq+9T3uvfUmeP9XPPu1/+DZbf2v/41tte+rAOccnuTjJt5PckOStSe604j6npVfW9Yb2vIszUBbTZltrWyVE6W+2kW5J/l0bvG9Mbyrfr6w4XknOS2/q343pVbG6z4pzbpvkj5Psb7c/TnLraX822+xtSU5I73l330jyvSRfTfLbSbYNnKPP2Ta9JXnUGr+7XdoeH0n/Sm/myPvbe1yZ5DczxfLmTdP03hwAAIBurIkCAAAYghAFAAAwBCEKAABgCEIUAADAEIQoAACAIQhRAAAAQxCiAAAAhiBEAQAADEGIAmBhVdXeqnrutNsBwGIRogCYe1X11Kr67iqHHpTk9yfw/sIawBI5dtoNAIBxaZrm2mm3YRhVta1pmsPTbgcA6zMSBcDIVNV7q+r1VfVfq+o7VXVVVZ3X8dqdVfX7VXVNVe2vqn+oqvsPHL9/Vf3fqjrQHv9EVT2wqh6V5I+S7Kyqpt3Oa685YoSoPfb0qnpbVd1QVV+sqodV1elt26+vqo9U1Y8NXPNjVfW3VXV1VR2sqo9V1U8PfuYkd07y2v77Dxx7YlV9vqoOtW15/orPvLeqfqOqLq2qfUn+oKq2VdUlVXVlVd3YnnPuUP8iABgrIQqAUXtKkuuTPCTJC5O8rKoes94FVVVJ/i7JHZP8XJKzknwyyXuq6rbtaX+S5LL0puidleSVSb6f5MNJnptkf5KT2+2Cdd7upUnekOTMJF9K8qYk/z3JK5I8sD3nkoHzdyR5e5KfTvKAJO9I8taqOq09/oS2XS8beP9U1VlJ3pLkzUnum+S8JC+vqqeuaM8Lknyu/UwvT/LsJD+f5BeT3D3JLyfZu87nAWDCTOcDYNQ+0zTNb7Xff6Wqnpnk0Unetc41P5le0Lh90zSH2n3nVNXjkvz79NY1nZbk1U3TfKl/7/7F7ShO0zTNVR3a90dN07ylve5VST6S5OVN07yj3fe69Ea2kt5NP53k0wPX/0ZVPT69oHNJ0zTfqaqbkhxY8f7PS/Kepmle3r7+clXdK73QdOnAef/QNM3/D31tOPtKkg82TdMk+UaHzwTABBmJAmDUPrPi9ZVJbr/BNWelN+Lz7XbK3MGqOpjkrkn6U+tek+QPq+rdVfXiwSl3W2jf1e3Xz67Yd3xVnZgkVbW9nZ74har6btuue6QX6tZzzyQfWrHvQ0nuVlXHDOz7+IpzLk1vlOyf26mRj93wEwEwUUIUAKP2/RWvm2z88+YW6YWtM1dsd0/y6iRpmua8JPdOb9rfTyX5QjsitJX2Nevs67f51UmemOTXkzyibddnk2zb4H1q4F6D+1a6fvBF0zSfTC88vjTJLZO8par+YoP3AmCCTOcDYBZ8Mr31UD9ommbvWic1TfPlJF9Or4jDnyZ5WpK/TnI4yTFrXbdFj0hyadM0f50kVbUjyV1WnLPa+38hyY+v2PfwJF9umuam9d6waZr9Sf4syZ+1Aer/VNVtm6b5zuY+AgCjZCQKgFnw7vTWJv1NVf1MVd2lqh5eVb/dVuC7ZVux7lFVdeeq+tfpFZj4Ynv93iQ7qurRVXVSVd1qhG3bk+QJVXVmWy3wTTn65+feJI+sqlOq6qR234VJHl1VL62qM6rqKUmemfWLXqSqzq6qJ1XVParqjCS/kOSqJKs9BwuAKRCiAJi6toDCzyV5f5L/md5o05vTG/G5OslNSW6XXlW9L6dX9e7vk/xme/2Hk/xeeqM316ZXFXBUzk5yXXpVAN+aXnW+T64452VtW7/avn9/Wt4vJnlSetX3/nOSlzVNc+kG73cwyYvSWyv1sfa+P9c0zQ+3/EkAGInq/dwCAACgCyNRAAAAQxCiABi7qnryYOnyFdvnp90+ABiG6XwAjF1VnZDkDmsc/n7TNB4oC8DcEKIAAACGYDofAADAEIQoAACAIQhRAAAAQxCiAAAAhiBEAQAADEGIAgAAGIIQBQAAMIT/BwrQN5FbML92AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x24181057780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('my_preds4_2_3_699.csv')\n",
    "\n",
    "cvresult = cvresult.iloc[100:]\n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(100,cvresult.shape[0]+100)\n",
    "        \n",
    "fig = pyplot.figure(figsize=(10, 10), dpi=100)\n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators_detail4_2_3_699.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "找到max_depth和min_child_weight的最佳参数后再一次对n_estimators进行参数调整，调整结果依然为999，与之前结果相同，该处没有提高可能是因为n_estimators数值设置较小。\n",
    "根据上图所示，并与之前的图进行比较，发现图中所示的标准差有明显减小，可以推测模型性能有所提高。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第三步：调整树的参数：subsample 和 colsample_bytree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': [0.6, 0.7, 0.8, 0.9],\n",
       " 'subsample': [0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "subsample = [i/10.0 for i in range(3,9)]\n",
    "colsample_bytree = [i/10.0 for i in range(6,10)]\n",
    "param_test3_1 = dict(subsample=subsample, colsample_bytree=colsample_bytree)\n",
    "param_test3_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.22368, std: 0.00476, params: {'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  mean: -0.21558, std: 0.00404, params: {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  mean: -0.21045, std: 0.00435, params: {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  mean: -0.20817, std: 0.00504, params: {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  mean: -0.20581, std: 0.00451, params: {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  mean: -0.20640, std: 0.00445, params: {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  mean: -0.21939, std: 0.00416, params: {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  mean: -0.21179, std: 0.00461, params: {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  mean: -0.20736, std: 0.00503, params: {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  mean: -0.20515, std: 0.00456, params: {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  mean: -0.20327, std: 0.00490, params: {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  mean: -0.20347, std: 0.00510, params: {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  mean: -0.21698, std: 0.00413, params: {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  mean: -0.20922, std: 0.00565, params: {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  mean: -0.20571, std: 0.00473, params: {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  mean: -0.20373, std: 0.00495, params: {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  mean: -0.20210, std: 0.00510, params: {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  mean: -0.20183, std: 0.00470, params: {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  mean: -0.21478, std: 0.00475, params: {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  mean: -0.20826, std: 0.00554, params: {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  mean: -0.20557, std: 0.00505, params: {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  mean: -0.20291, std: 0.00468, params: {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  mean: -0.20095, std: 0.00557, params: {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  mean: -0.20098, std: 0.00522, params: {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       " -0.2009509017986338)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb3_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=999,  #第二轮参数调整得到的n_estimators最优值\n",
    "        max_depth=9,\n",
    "        min_child_weight=0,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch3_1 = GridSearchCV(xgb3_1, param_grid = param_test3_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch3_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch3_1.grid_scores_, gsearch3_1.best_params_,     gsearch3_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([5024.57924757, 5235.46689081, 5338.37824922, 5331.56771312,\n",
       "        5182.6441433 , 5093.39215131, 5119.69517837, 5628.94044428,\n",
       "        5989.89138751, 5700.36786327, 5450.19520459, 5280.97445135,\n",
       "        5309.90576768, 5881.56450419, 6260.93383951, 6360.30927215,\n",
       "        6232.42604866, 6088.04338002, 5918.60932364, 6562.90556169,\n",
       "        7133.62629037, 7188.94299803, 7075.8600667 , 6734.91020966]),\n",
       " 'mean_score_time': array([87.52356653, 64.06240497, 59.05849109, 61.53873477, 60.11679554,\n",
       "        61.41145043, 62.3076438 , 61.40544667, 58.38284345, 58.23894973,\n",
       "        55.41768584, 55.65043645, 57.44082232, 53.79280858, 57.2366837 ,\n",
       "        57.27130947, 59.52500114, 54.63996816, 59.33827815, 71.97362928,\n",
       "        60.79303799, 61.91938591, 65.7249032 , 48.50413866]),\n",
       " 'mean_test_score': array([-0.22368114, -0.21557683, -0.21045035, -0.20817147, -0.20581127,\n",
       "        -0.20639661, -0.21939205, -0.21179331, -0.20736228, -0.20514917,\n",
       "        -0.20327242, -0.20346665, -0.21698345, -0.20922431, -0.20571068,\n",
       "        -0.20372623, -0.20210123, -0.20183326, -0.21478061, -0.20825574,\n",
       "        -0.20556974, -0.20290505, -0.2009509 , -0.20098123]),\n",
       " 'mean_train_score': array([-0.05046645, -0.0452388 , -0.04297843, -0.04218839, -0.04215381,\n",
       "        -0.04341515, -0.04567743, -0.04110071, -0.03924456, -0.03833954,\n",
       "        -0.03862556, -0.03975727, -0.04252045, -0.03842601, -0.03672292,\n",
       "        -0.03620347, -0.03660714, -0.03798831, -0.03982616, -0.03636844,\n",
       "        -0.03489667, -0.0345637 , -0.03502503, -0.03632519]),\n",
       " 'param_colsample_bytree': masked_array(data=[0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.7, 0.7, 0.7, 0.7, 0.7,\n",
       "                    0.7, 0.8, 0.8, 0.8, 0.8, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9,\n",
       "                    0.9, 0.9],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_subsample': masked_array(data=[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6, 0.7,\n",
       "                    0.8, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6,\n",
       "                    0.7, 0.8],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " 'rank_test_score': array([24, 21, 18, 15, 12, 13, 23, 19, 14,  9,  6,  7, 22, 17, 11,  8,  4,\n",
       "         3, 20, 16, 10,  5,  1,  2]),\n",
       " 'split0_test_score': array([-0.22533777, -0.21627565, -0.21090213, -0.20723755, -0.20636637,\n",
       "        -0.20655937, -0.21757905, -0.21152486, -0.20818636, -0.20530149,\n",
       "        -0.20265815, -0.2041614 , -0.21649076, -0.20846372, -0.2022192 ,\n",
       "        -0.205059  , -0.2012633 , -0.20100629, -0.21351931, -0.20741206,\n",
       "        -0.2052565 , -0.20384842, -0.20012987, -0.20194112]),\n",
       " 'split0_train_score': array([-0.05070352, -0.04481848, -0.0427086 , -0.04179561, -0.04126626,\n",
       "        -0.04282209, -0.04569435, -0.0407483 , -0.0388806 , -0.03823683,\n",
       "        -0.03851874, -0.04000744, -0.04233313, -0.03827923, -0.03649996,\n",
       "        -0.0359635 , -0.03589818, -0.03823912, -0.03935768, -0.03629934,\n",
       "        -0.0345583 , -0.03404754, -0.03449124, -0.03598048]),\n",
       " 'split1_test_score': array([-0.23188855, -0.22262813, -0.21850442, -0.21766134, -0.21388481,\n",
       "        -0.21466452, -0.22697167, -0.21986242, -0.21621749, -0.21326615,\n",
       "        -0.21249152, -0.21269626, -0.22397464, -0.22007873, -0.214908  ,\n",
       "        -0.21221653, -0.21155699, -0.2105226 , -0.22392309, -0.21852739,\n",
       "        -0.21521148, -0.21162718, -0.21115269, -0.21043484]),\n",
       " 'split1_train_score': array([-0.05092521, -0.0449521 , -0.04273242, -0.0432457 , -0.04321741,\n",
       "        -0.04393089, -0.04529488, -0.04103511, -0.03923079, -0.03814265,\n",
       "        -0.03777812, -0.03948545, -0.04264463, -0.03878964, -0.03700908,\n",
       "        -0.03658037, -0.03713738, -0.0381281 , -0.03947415, -0.03593274,\n",
       "        -0.03502374, -0.03459139, -0.03604176, -0.03665179]),\n",
       " 'split2_test_score': array([-0.2217603 , -0.21250325, -0.20828979, -0.20466173, -0.20472981,\n",
       "        -0.20348777, -0.2183332 , -0.21074792, -0.20469201, -0.20178021,\n",
       "        -0.20114722, -0.20057098, -0.21778932, -0.20801387, -0.20493864,\n",
       "        -0.20147465, -0.20059537, -0.19866069, -0.21373541, -0.20800296,\n",
       "        -0.20372331, -0.20022854, -0.2003096 , -0.19851689]),\n",
       " 'split2_train_score': array([-0.05023764, -0.04443779, -0.04301179, -0.04148767, -0.04168336,\n",
       "        -0.04243656, -0.04520869, -0.04075166, -0.03900714, -0.0376406 ,\n",
       "        -0.03851916, -0.03964711, -0.04230024, -0.03782592, -0.03654502,\n",
       "        -0.03587158, -0.0365501 , -0.0372372 , -0.03935233, -0.03628928,\n",
       "        -0.03501189, -0.03430714, -0.03465613, -0.03588619]),\n",
       " 'split3_test_score': array([-0.22168416, -0.21557409, -0.20878219, -0.20800975, -0.20385417,\n",
       "        -0.20552421, -0.2196317 , -0.2113326 , -0.2066469 , -0.20539909,\n",
       "        -0.20216564, -0.20231047, -0.21542287, -0.20514668, -0.20431153,\n",
       "        -0.20268408, -0.20102194, -0.20207382, -0.21262866, -0.20528383,\n",
       "        -0.20310899, -0.20003136, -0.19898787, -0.1990086 ]),\n",
       " 'split3_train_score': array([-0.05058991, -0.04555124, -0.04266789, -0.04211427, -0.04183319,\n",
       "        -0.04387819, -0.04586646, -0.0416483 , -0.03957606, -0.03847679,\n",
       "        -0.03940058, -0.03968821, -0.04261283, -0.0383574 , -0.03626413,\n",
       "        -0.03601267, -0.03665667, -0.03812784, -0.0402591 , -0.03684274,\n",
       "        -0.03466943, -0.0347277 , -0.03450993, -0.03652276]),\n",
       " 'split4_test_score': array([-0.21773381, -0.21090214, -0.20577232, -0.20328601, -0.20022017,\n",
       "        -0.20174623, -0.2144438 , -0.2054977 , -0.20106749, -0.19999793,\n",
       "        -0.19789855, -0.19759303, -0.21123874, -0.20441753, -0.2021754 ,\n",
       "        -0.19719572, -0.19606751, -0.196902  , -0.21009568, -0.20205133,\n",
       "        -0.2005474 , -0.19878882, -0.19417329, -0.19500355]),\n",
       " 'split4_train_score': array([-0.04987599, -0.04643437, -0.04377144, -0.04229869, -0.04276884,\n",
       "        -0.04400804, -0.04632275, -0.04132017, -0.03952819, -0.03920083,\n",
       "        -0.0389112 , -0.03995815, -0.04271141, -0.03887784, -0.03729642,\n",
       "        -0.03658923, -0.03679337, -0.03820929, -0.04068754, -0.0364781 ,\n",
       "        -0.03521999, -0.03514471, -0.03542608, -0.03658474]),\n",
       " 'std_fit_time': array([ 43.65492046, 160.34990112,  24.84292494,  57.32580447,\n",
       "         11.72112259,  40.14366027,  12.1800237 ,  43.42619709,\n",
       "        104.97880883, 115.20378049,  29.43211635,  16.35266602,\n",
       "         17.1822271 ,  22.86439238,  53.38613959,  37.56644476,\n",
       "         26.73269281,  94.94559479,  35.20931377,  21.18204538,\n",
       "         28.72235448,  24.27056126,  17.45977434, 274.86722281]),\n",
       " 'std_score_time': array([15.16308971,  6.5995355 ,  3.13909808,  1.56846948,  3.36394886,\n",
       "         2.18474726,  2.27482537,  2.22207206,  3.00955817,  4.19084449,\n",
       "         3.43912866,  1.79305459,  3.42013279,  0.96452768,  6.3595224 ,\n",
       "         2.15858876,  2.41196814,  0.36220194,  5.02265082, 25.02922606,\n",
       "         2.79961765,  2.14117115,  9.19999152, 13.62413899]),\n",
       " 'std_test_score': array([0.00475719, 0.00403755, 0.00434529, 0.0050427 , 0.00451111,\n",
       "        0.00445499, 0.00415697, 0.00460699, 0.00502713, 0.00455756,\n",
       "        0.00489878, 0.00509687, 0.00412808, 0.00564975, 0.00472941,\n",
       "        0.00495306, 0.00509809, 0.00470384, 0.00475078, 0.00554276,\n",
       "        0.00505474, 0.00467617, 0.0055669 , 0.00521517]),\n",
       " 'std_train_score': array([0.00036957, 0.00069678, 0.00041466, 0.0005968 , 0.00072407,\n",
       "        0.00065441, 0.0004046 , 0.00034593, 0.00027545, 0.00050961,\n",
       "        0.0005335 , 0.00019684, 0.00016971, 0.00038019, 0.00037487,\n",
       "        0.00031464, 0.000406  , 0.00037813, 0.00054723, 0.00029593,\n",
       "        0.00024499, 0.00037349, 0.00061334, 0.00032392])}"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch3_1.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.200951 using {'colsample_bytree': 0.9, 'subsample': 0.7}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XdUVNfax/HvHnpHqliwNwREYzcW7KgxUYzGXhNNYos3JhqTa8ybek2x18Ro1MTEHnvvGrtiQcUKKKCCgID0/f4xSFCR4swAwv6sxbowc8qeuWZ+c84+53mElBJFURRFeVGawh6AoiiK8nJTQaIoiqLoRAWJoiiKohMVJIqiKIpOVJAoiqIoOlFBoiiKouhEBYmiKIqiExUkiqIoik5UkCiKoig6MS7sARQEJycnWbFixcIehqIoykvl5MmT96WUzrktVyKCpGLFipw4caKwh6EoivJSEULcysty6tSWoiiKohMVJIqiKIpOVJAoiqIoOikRcySKUhSkpKQQGhpKYmJiYQ9FUZ5gbm5OuXLlMDExeaH1VZAoSgEJDQ3FxsaGihUrIoQo7OEoCgBSSiIjIwkNDaVSpUovtA11aktRCkhiYiKOjo4qRJQiRQiBo6OjTkfKKkgUpQCpEFGKIl3/XaogycGuwAj+OhFS2MNQFEUp0tQcyXNIKVl+NJh9V+5R2tacFtVzvblTURSlRFJHJM8hhGD6Wz5Uc7HmveWnuBQeW9hDUhSdRUdHM2fOnBdad9q0aSQkJOhtLIsXL2bkyJF6215e7d27ly5duuR7vfyO9+bNm/z+++/53k9eRUVF0a5dO6pVq0a7du148OBBtssFBwfTvn17atWqhYeHBzdv3tT7WFSQ5MDG3IRfBzfAysyIIb8eJyJWXbapvNyKUpAUdzkFSWpqqs7b//bbb2nTpg1BQUG0adOGb7/9NtvlBgwYwPjx4wkMDOTYsWO4uLjovO+nqVNbuXCzs+CXgQ3oOf8IQxYf56/hTbAyU2+bopspGy5w8Y5+j3I9ytgy+bXaOS4zYcIErl27ho+PD+3atcPFxYW//vqLpKQkunXrxpQpU4iPj6dnz56EhoaSlpbGZ599RkREBHfu3MHX1xcnJyf27NmT7fa3bt3KJ598QlpaGk5OTuzatYuoqCiGDBnC9evXsbS0ZMGCBXh7ez+x3sqVK5kyZQpGRkbY2dmxf/9+bt68Sf/+/YmPjwdg1qxZNG3alL179zJ58mRcXV05c+YM3bt3x8vLi+nTp/Po0SPWrVtHlSpVGDRoEObm5ly4cIGIiAh+/PHHZ45E4uPjGTVqFOfOnSM1NZXPP/+c119//bnvX0hICB07duTGjRv06dOHyZMn89lnn+Hk5MSYMWMAmDRpEq6urvz+++8EBgbi4+PDwIEDKVWqFJs2bSIxMZH4+Hh2797N1KlTn3n/AZYtW8aMGTNITk6mUaNGzJkzByMjoyfGsn79evbu3QvAwIEDadWqFd99990Ty1y8eJHU1FTatWsHgLW1dU7/PF6Y+kTMA8+ydszuU4+hS44z+o/TLBhQHyONuvpGefl8++23nD9/njNnzrB9+3ZWrVrFsWPHkFLStWtX9u/fz7179yhTpgybNm0CICYmBjs7O3788Uf27NmDk5NTttu+d+8eb7/9Nvv376dSpUpERUUBMHnyZOrWrcu6devYvXs3AwYM4MyZM0+s+8UXX7Bt2zbKli1LdHQ0AC4uLuzYsQNzc3OCgoLo3bt3ZvHVs2fPEhgYiIODA5UrV2bYsGEcO3aM6dOnM3PmTKZNmwZojwr27dvHtWvX8PX15erVq0/s96uvvqJ169YsWrSI6OhoGjZsSNu2bbGyssr2NR47dozz589jaWlJgwYN6Ny5M0OHDqV79+6MGTOG9PR0VqxYwbFjx/D29ub7779n48aNgPbU2JEjRwgICMDBwYHt27cTFBT0zPvv7OzMn3/+yaFDhzAxMeG9995j+fLlDBgwgGHDhjFixAjq169PREQEbm5uALi5uXH37t1nxnvlyhXs7e3p3r07N27coG3btnz77bfPhJKuVJDkkW9NF6Z0rc1n6y/wxYYLfN61trqUU3lhuR05FITt27ezfft26tatC0BcXBxBQUE0b96cDz/8kI8//pguXbrQvHnzPG3vn3/+oUWLFpk3tTk4OABw8OBBVq9eDUDr1q2JjIwkJibmiXWbNWvGoEGD6NmzJ927dwe0lQBGjhzJmTNnMDIy4sqVK5nLN2jQIPNDtEqVKrRv3x4ALy+vJ46WevbsiUajoVq1alSuXJlLly498x78/ffffP/994D2Xp/g4GBq1aqV7Wts164djo6OAHTv3p2DBw8yduxYHB0dOX36NBEREdStWzdzmezWf/y+PO/9DwgI4OTJkzRo0ACAR48eZZ6O+vnnn7Pd7vOkpqZy4MABTp8+jbu7O7169WLx4sUMHTo0X9vJjQqSfOjfpCK3IhP4+eAN3B2tGPrqi90FqihFgZSSiRMnMnz48GeeO3nyJJs3b2bixIm0b9+e//73v3naXnZfrqSUzzz29HLz5s3j6NGjbNq0CR8fH86cOcPMmTNxdXXl7NmzpKenY25unrm8mZlZ5u8ajSbzb41G88T8w9P7efpvKSWrV6+mRo0aub6+nLY3bNgwFi9eTHh4OEOGDHnu+lmPdJ73/s+cOZOBAwfyzTff5DgWV1dXwsLCcHNzIywsLNu5j3LlylG3bl0qV64MwBtvvME///yj9yBRk+359EmnWnSo7cqXmy6y7UJ4YQ9HUfLFxsaGhw8fAtChQwcWLVpEXFwcALdv3+bu3bvcuXMHS0tL+vXrx4cffsipU6eeWTc7TZo0Yd++fdy4cQMg89RWixYtWL58OaC9YsrJyQlbW9sn1r127RqNGjXiiy++wMnJiZCQEGJiYnBzc0Oj0bB06VLS0tLy/XpXrlxJeno6165d4/r1688ERocOHZg5c2Zm2J0+fTrH7e3YsYOoqKjMuZhmzZoB0K1bN7Zu3crx48fp0KEDkPv79bz3v02bNqxatSrzVFVUVBS3bj3bFqRr164sWbIEgCVLlmQ7t9OgQQMePHjAvXv3ANi9ezceHh45vsYXoY5I8kmjEUzrVZe3Fv7DmBWn+fOdJtQpb1/Yw1KUPHF0dKRZs2Z4enri5+dHnz59aNKkCaCdiF22bBlXr15l/PjxaDQaTExMmDt3LgDvvPMOfn5+uLm5ZTvZ7uzszIIFC+jevTvp6emZcxyff/45gwcPxtvbG0tLy8wPv6zGjx9PUFAQUkratGlDnTp1eO+99/D392flypX4+vo+d94iJzVq1KBly5ZEREQwb968J45qAD777DPGjh2Lt7c3UkoqVqyYOaeRnVdffZX+/ftz9epV+vTpQ/369QEwNTXF19cXe3v7zPkHb29vjI2NqVOnDoMGDaJUqVJPbKt9+/YEBgY+8/57eHjw5Zdf0r59e9LT0zExMWH27NlUqFDhiTmSCRMm0LNnT3755Rfc3d1ZuXIlACdOnGDevHn8/PPPGBkZ8f3339OmTRuklLzyyiu8/fbb+X4fcyOyO+wsburXry/13SHx3sMkus05RGJKOmvfa0p5B0u9bl8pfgIDA5977l3Rv0GDBtGlSxd69Ohh8H2lp6dTr149Vq5cSbVq1Qy+P0PI7t+nEOKklLJ+buuqU1svyNnGjMWDG5CcmsaQxceJeZRS2ENSFKUQXLx4kapVq9KmTZuXNkR0pU5t6aCqiw3z+r/CwEXHeHfZSRYPboipscpmpfhr1KgRSUlJTzy2dOlSvLy8CmlEz1q8ePELrbdt2zY+/vjjJx6rVKkSa9euzXZ5Dw8Prl+//kL7Ki5UkOioaRUnvunuzYcrz/LJ2nNM7eGtLgtWir2jR48W9hAMpkOHDpkT5kreqCDRgx6vlCM4KoEZu4Ko4GDJqDYl8/BWUZSSSQWJnnzQthohUQn8sOMK7o6WvO5TtrCHpCiKUiBUkOiJEIJv/b24Hf2I8SsDcLOzoGElh8IelqIoisGpmWE9MjM2YkH/VyjnYME7S09w/V5cYQ9JURTF4FSQ6Jm9pSmLBzXESAgGLz5OZFxS7ispSgEpSmXkVT8S3eSlH8mePXvw8fHJ/DE3N2fdunV6H4sKEgNwd7Rk4cD6hMck8s7SkySm5L+0g6IYQlEKkuKuKPQj8fX15cyZM5w5c4bdu3djaWmZWeBSn9QciYHUcy/FT718eG/5Kf6z8iwz36qLRpWeVx7bMgHCz+l3m6W9wC/75kaPqX4kJasfSVarVq3Cz88PS0v9V+FQQWJAnbzcmOhXk2+2XKJ8KUsm+NUs7CEpJZzqR1Ky+pFktWLFCsaNG5fjMi9KBYmBvdOiMreiEpi37xoVHC3p3dC9sIekFAW5HDkUBNWPpPj3I3ksLCyMc+fOGexGSxUkBiaE4Iuutbn94BGfrjtPGXsLWlZ3LuxhKYrqR0Lx70fy2F9//UW3bt0wMTHJcZsvSk22FwBjIw2z+tSlmos17y8/RWCYfnt1K0peqX4kJasfyWN//PEHvXv3zvG16UIdkRQQG3MTfh3cgDdmH2LI4uOse78Zrrbmua+oKHqk+pGUrH4koJ0nCgkJoWXLlvl+//JK9SMpYOdvx9Bz/hEqOVnx1/AmWJmpLC8pVD+SgqX6keSP6kfyEvEsa8fsPvUIDItl9B+nSUsv/kGuKMWZ6keiTm0VCt+aLkx53ZPP1p3niw0X+LxrbVV6XnmpqH4k/1L9SFSQFJr+jSsQHBnPwgM3cHe0YuirlQp7SIqSZ6ofiZKVQU9tCSE6CiEuCyGuCiEmZPP8OCHERSFEgBBilxCiQsbjPkKII0KICxnP9cqyTiUhxFEhRJAQ4k8hhKkhX4MhTfSrRcfapfly00W2ng8v7OEoiqK8EIMFiRDCCJgN+AEeQG8hhMdTi50G6kspvYFVwP8yHk8ABkgpawMdgWlCCPuM574DfpJSVgMeAEMN9RoMTaMR/NTLB+9y9oz98zRnQqILe0iKoij5ZsgjkobAVSnldSllMrACeOJCZynlHinl4ypw/wDlMh6/IqUMyvj9DnAXcBbaiYTWaEMHYAnwhgFfg8FZmBrx84D6OFmbMWzJcUKiVFE8RVFeLoYMkrJASJa/QzMee56hwJanHxRCNARMgWuAIxAtpXx86+pztymEeEcIcUIIceLevXsvMPyC42xjxuLBDUhOTWfw4uPEPEop7CEpxVRRqv6rysjrJi9l5AE++ugjateuTa1atRg9enS2lQZ0Zcggye4ypGxfgRCiH1AfmPrU427AUmCwlDI9P9uUUi6QUtaXUtZ3di76JUmqutgwr/8r3IqM591lJ0lOTS/sISnFUFEKkuKuKJSRP3z4MIcOHSIgIIDz589z/Phx9u3bp/O+n2bIIAkFymf5uxxw5+mFhBBtgUlAVyllUpbHbYFNwKdSyn8yHr4P2AshHl9tlu02X1ZNqzjxbXdvDl+L5JO15wzyzUEp2bKWkR8/fjxTp06lQYMGeHt7M3nyZEBbWr1z587UqVMHT09P/vzzT2bMmJFZRt7X1/e529+6dSv16tWjTp06tGnTBtB+c37jjTfw9vamcePGBAQEPLPeypUr8fT0pE6dOrRo0QLQfhA3b96cevXqUa9ePQ4fPgxojyhatmxJz549qV69OhMmTGD58uU0bNgQLy8vrl27BmhvSBwxYgTNmzenevXq2d6xHh8fz5AhQ2jQoAF169Zl/fr1Ob5/j8vI16hRI7Pk+2effcb06dMzl5k0aRIzZsxgwoQJHDhwAB8fH3766ScWL17Mm2++yWuvvZZZZDK79x+0ZeQbNmyIj48Pw4cPz7Y8zPr16xk4cCCgLSOfXcMqIQSJiYkkJyeTlJRESkoKrq6uOb7GF2HIy3+PA9WEEJWA28BbQJ+sCwgh6gLzgY5SyrtZHjcF1gK/SSlXPn5cSimFEHuAHmjnXAYCOf8//5Lxf6UcwVEJTN8VRAUHS0a1KZk3OBV33x37jktRl3JfMB9qOtTk44Yf57iMKiNfssrIN2nSBF9fX9zc3JBSMnLkSINUVzBYkEgpU4UQI4FtgBGwSEp5QQjxBXBCSvk32lNZ1sDKjBvygqWUXYGeQAvAUQgxKGOTg6SUZ4CPgRVCiC/RXvX1i6FeA7F3IDEWXAq2j8jYttUIiUrghx1XcHe05HWfnKaWFOXFqDLyxb+M/NWrVwkMDCQ0NDRz//v378886tMXg96QKKXcDGx+6rH/Zvm97XPWWwYse85z19FeEWZYUsLKQfDgFgzdBqUqGnyXjwkh+Mbfi9vRjxi/MoDStuY0qpz9P0zl5ZTbkUNBUGXki38Z+bVr19K4cWOsra0B8PPzywx8fVK1tp5HCOgyDVITYWk3iMu5+5i+mRkbsaB/fco5WPDO0pNcuxdXoPtXiidVRr5klZF3d3dn3759pKamkpKSwr59+wxyaksFSU5cPaDvSngYDsu6Q2JM7uvokZ2lCYsHNcRYIxj863Ei45JyX0lRcpC1jPyOHTsyy8h7eXnRo0cPHj58yLlz5zIner/66is+/fRT4N8y8s+bbM9aRr5OnTr06qUtSPH5559z4sQJvL29mTBhwnPLyHt5eeHp6UmLFi0yy8gvWbKExo0bc+XKFZ3KyPv5+T23jHxKSgre3t54enry2Wef5bi9x2XkfXx88Pf3f6aMfM+ePbMtI//TTz89s6327dtn+/5nLSPv7e1Nu3btCAsLA7RHPo/niSZMmMCOHTuoVq0aO3bsYMIEbfGQEydOMGzYMAB69OhBlSpV8PLyok6dOtSpU4fXXnst3+9jblQZ+by4uhN+fwvKN4R+q8HEQn+Dy4NTwQ/oveAfapex5fe3G2NuYlSg+1f0Q5WRL1iqjHz+qDLyhla1LXSbB7cOw6ohkKb7NeD5Uc+9FD/18uFUcDT/+ess6ar0vKIUGaqMvKr+m3dePeDRA9j8Ifw9Cl6fDZqCy+FOXm580qkmX2++RHkHSyb4FeyVZIqSlSoj/y9VRl4FSf40fBsSomDv12DpAO2/1E7KF5C3m1fmVmQC8/Zdo4KjJb0buhfYvhUlK1VGXslKBUl+tfwIEiLhyCywdITm4wps10IIpnStze3oR3y67jxl7C1oWb3ol39RFKV4U3Mk+SUEdPwWvN6EXVPg5OIC3b2xkYZZfepR3dWG95efIjAstkD3ryiK8jQVJC9Co4E35kLVdrDxA7hYsFVarM2MWTSoPlZmRgxZfJyI2MQC3b+iKEpWKkhelJEJ9PwNyjWE1cPg+t4C3b2bnQWLBjUg9lEKQxYfJz6pYK8kUxRFeUwFiS5MLaHPCnCsBiv6wu2TBbr72mXsmNWnHoFhsYz64zSpaar0vJKzolRGXvUj0U1e+5F8/PHHeHp6ZlZyNgQVJLqyKKW9SdHSEZb1gHtXcl9Hj3xrujDldU92X7rLFxsvqtLzSo6KUpAUd0WhH8mmTZs4deoUZ86c4ejRo0ydOpXYWP3Pq6og0QdbN+i/FjTGsPQNiA7JfR096t+4Am83r8RvR27xy8EbBbpv5eWi+pE8qbj3I7l48SItW7bE2NgYKysr6tSpw9atW3N8jS9CXf6rL45VoP8a+LWztsjjkK1glX3fBkOY6FeL0AeP+GpzIOVKWdLRs3SB7VvJv/CvvyYpUL/9SMxq1aT0J5/kuIzqR1Ky+pHUqVOHKVOmMG7cOBISEtizZw8eHh45/ht5ESpI9Km0l3bOZGk3WN4DBm4AM5sC2bVGI/iplw9hC/5h7J+nWWHXBJ/y9gWyb+XlpPqRFP9+JO3bt+f48eM0bdoUZ2dnmjRpgrGx/j/2VZDoW4Wm8OYSWNFH+9N3FRib5b6eHpibGPHzwPp0m3OIYUuOs/a9ZpR3sCyQfSv5k9uRQ0FQ/UiKfz8S0J5qmzRpEgB9+vQxSD0wNUdiCDU6amtx3divvTQ4Pf99FF6Uk7UZvw5qQHJqOoMXHyfmUUqB7Vsp+lQ/kpLVjyQtLY3IyEgAAgICCAgIyDx60yd1RGIoPr21RR63TdTetPja9AKry1XVxYb5/eszYNFRRiw9yZIhDTE1Vt8ZlCf7kfj5+WX2wwCwtrZm2bJlXL16lfHjx6PRaDAxMWHu3LnAv/1I3Nzcnjh99FjWfiTp6emZcxyff/45gwcPxtvbG0tLy+f2IwkKCkJKSZs2bTL7kfj7+7Ny5Up8fX116kcSERHx3H4kY8eOxdvbGyklFStWzHZS/rHH/UiuXr1Knz59nulHYm9vn20/kkGDBlGqVKknttW+fXsCAwOfef+z9iNJT0/HxMSE2bNnU6FChSfmSCZMmEDPnj355ZdfcHd3Z+XKlYC2H8m8efP4+eefSUlJyTw1aWtry7Jlywxyakv1IzG0Xf8HB76HV8dB28m5L69Hq0+G8p+VZ/GvV47v3/TO9rSDUnBUP5KCpfqR5I/qR2IgIQ9DuBB5QbeNtP4UXhkMB3+EwzP1M7A88n+lHGPaVGP1qVBm7r6a+wqKouSb6keiTm09l5SSTw58wq3YW/zm9xsV7Sq+2IaEgM4/aE9zbf8ULBygbl+9jjUnY9tWIyQqgR93XMHdwZI36pYtsH0rxZfqR/Iv1Y9EndrK0a3YWwzYMgALYwuW+i3F2VKHku2pSfB7T7hxAHotg5qdXnxb+ZScms6ARUc5dSuapUMb0qhy9pcmKoalTm0pRZk6tWUgFWwrMKfNHKISo3h357s8TH7+FRi5MjaDXsuhjA+sHAQ3D+ptnLkxNdYwv199yjtY8M7Sk1y7F1dg+1aeVBK+uCkvH13/XaogyUVtp9pMazWNa9HXGLNnDMlpyS++MTNr6LMSSlWEP3pD2Fm9jTM3dpYm/DqoIcYaweBfjxMZl5T7SopemZubExkZqcJEKVKklERGRj5zRVt+qFNbebTx+kYmHphI+wrt+V+L/2GkMXrxjcXchkUdIDURhmzTllcpIKeDH/DWgn+oXcaW399ujLmJDq9DyZeUlBRCQ0NJTFT9Y5SixdzcnHLlymFiYvLE43k9taWCJB+WXFjC9ye+p3fN3kxsOFG3y2nvB2nDxMQKhm4D2zI6jy+vtpwL473fT+HnWZpZveuh0ajLghVFeZaaIzGAgbUHMtBjIH9c+oOfz+Wv5s0znKppy88/ioKl3SEhSj+DzAM/Lzcm+tVk87lwvtum38KBiqKUPCpI8mlc/XF0rtyZGadnsDYo+8sB86xMXXjrd4i6pr2iKzleP4PMg7ebV6ZvI3fm77vO70eDC2y/iqIUPypI8kkjNPxf0/+jaZmmTDkyhX0h+3TbYOWW4P+LtrviXwMgVYfJ/HwQQjCla21a1XDms/Xn2Xv52RLUiqIoeaGC5AWYGJnwU6ufqOlQkw/3fciZu2dyXyknHl21tbiu7oR1IyC9YFrmGhtpmNWnHtVdbRj5+2kCw/TfOU1RlOJPBckLsjSxZHab2bhYujBy90iuR+t4Z2u9AdB2CpxfDVs+ggK6CMLazJhFg+pjbWbMkMXHiYhVVxQpipI/Kkh04GjhyLx28zAWxgzfOZzw+HDdNvjqWGg6Co4vhL3P9l82FDc7CxYNakDsoxSGLD5OfJLu/aQVRSk5VJDoqLxNeea2ncvD5Ie8u/NdYpJicl8pJ+3+D3z6wb5v4eh8/QwyDzzK2DKrbz0uhT+k/y9HuR39qMD2rSjKy00FiR7UcqzFdN/p3Iq9xejdo0lM1eH0kBDa+ZIanbWnuAJW6m+gufCt4cLM3nW5EhGH37T9bD0fVmD7VhTl5WXQIBFCdBRCXBZCXBVCTMjm+XFCiItCiAAhxC4hRIUsz20VQkQLITY+tc5iIcQNIcSZjB8fQ76GvGrk1oivm3/N6bunmXBgAmm6dEU0MoYei6DCq9rJ96Ad+htoLjp5ubFp9KtUdLJixLJTfLruHIkpBdfhUVGUl4/BgkQIYQTMBvwAD6C3EMLjqcVOA/WllN7AKuB/WZ6bCvR/zubHSyl9Mn50vGRKfzpW7MjHDT9mV/Auvjr6lW41lUzMofcf4Fob/uwPwUf1N9BcVHC0YtWIprzdvBLL/gnmjdmHuHpXh4KViqIUa4Y8ImkIXJVSXpdSJgMrgCeaCksp90gpEzL+/Acol+W5XcBL9+nVt1ZfhnoOZeWVlcw7O0+3jZnbQt/V2vIpv78JETo22coHU2MNkzp78OvgBtx7mESXmQdZcSxYFRxUFOUZhgySskBIlr9DMx57nqHAljxu+6uM02E/CSHMXnSAhjKm3hher/I6c87O4a/Lf+m2MWtnGLAOTCy1pVSibuhnkHnkW8OFLWOaU8+9FBPWnGPUH6eJTUwp0DEoilK0GTJIsqsEmO3XWSFEP6A+2tNZuZkI1AQaAA7Ax9ktJIR4RwhxQghx4t69e3kb8VNSHzwg5W7+7/gWQjC56WSal23OV0e/YlfwrhfafyZ7d+i/VlsteGk3eBih2/byycXWnKVDGzG+Qw22nA+n84wDnAmJLtAxKIpSdBkySEKB8ln+LgfceXohIURbYBLQVUqZa5MMKWWY1EoCfkV7Ci275RZIKetLKes7O+e/s6GUkttjxnKrX3+SQ0Pzvb6JxoTvW36Pp6MnH+37iJMRJ/O9jSe41IK+qyAuApb5w6OC/SA30gje963KX8Mbk54OPeYeZv6+a6Snq1NdilLSGTJIjgPVhBCVhBCmwFvA31kXEELUBeajDZE8ffUXQrhl/K8A3gDO63XU/+4Hl/+MIy0mhlu9+5AUFJTvbViaWDKrzSzKWJdh1O5RBD3I/zaeUL4B9FoK9y5pG2OlFPy9Hq9UcGDz6Oa083Dlmy2XGLT4OPceqiZZilKS5RokQogqj+chhBCthBCjhRD2ua0npUwFRgLbgEDgLynlBSHEF0KIrhmLTQWsgZUZl/JmBo0Q4gCwEmgjhAgVQnTIeGq5EOIccA5wAr7M86vNJ4s6daiw9DcAbvXrz6OAgHxvo5R5Kea3m4+FkQUjdo4gLE7HezOqtoXu8yH4CKwcDGkFP19hZ2nCnL71+PINT45ej8Rv+gEOBL3Y6UNFUV5+uTa2EkKcQTt/URFtKPwN1JBSdjL46PRE18ZWySEhBA+jlRtnAAAgAElEQVQeQlpUFOXmzMaqceN8b+PKgysM2jIIJ0snfuv4G/bmuWZxzo7/DJv+A95vwRtzQVM495ZeCo9l5O+nuXYvjhEtqzCuXXVMjNR9ropSHOizsVV6xtFFN2CalPIDwE3XAb5MTMuXp8Ly5ZiULUvI2+/wcOfOfG+jeqnqzGg9g9sPb/P+7vd5lKrjaakGw8B3EgSsgO2TCqzI49NqlrZlw8hXeatBeebuvUbP+UcIiUrIfUVFUYqNvARJihCiNzAQeHyXuUkOyxdLJq4uVFj6G2YetQgdM5bodevyvY36pevzXYvvOH//POP3jSc1XcfiiC3GQ6MR8M8cOPCDbtvSgYWpEd9092ZWn7pcjYij04wDbApQ5VUUpaTIS5AMBpoAX0kpbwghKgHLDDusosnI3p4KixZh2bABYRMmEvXbb/neRtsKbZnUaBL7QvfxxZEvdLvBTwjo8A149YTd/wcnFr34tvSgi3cZNo9pThVna97//RQT15zjUbIqr6IoxV2uQSKlvCilHC2l/EMIUQqwkVIWXI3zIkZjZUX5+fOxadeWiK+/4d7MWfkOg541ejKizgjWXl3LzNMzdRyQBt6YA9U6wMZxcCH/R0r6VN7BkpUjmjCiZRX+OBbM67MPcjn8pStQoChKPuTlqq29QghbIYQDcBb4VQjxo+GHVnRpTE0p+9NP2HXrxv3Zs4n4+htkPrsavlfnPXpU78HCcwtZHrhctwEZmcCbi6F8I1g9DK7t1m17OjIx0jDBrya/DWlIVHwyXWcdZPnRW6q8iqIUU3k5tWUnpYwFugO/SilfAdoadlhFnzA2xu2rL3EYOIAHS5cSNvETZGre5zyEEExqNAnf8r58d+w7tt7cqtuATC2hz5/gVB1W9INQHW+A1IMW1Z3ZPKY5DSs5MGnted7//RQxj1R5FUUpbvISJMYZNwH25N/JdgUQGg0uEybgPGY0MevXEzpmLOlJeb85z1hjzP9a/I+6LnX55MAnHAs7ptuALOyh/xqwcoLl/nDvsm7b0wMXG3OWDG7IBL+abL8QQafpBzh560FhD0tRFD3KS5B8gfb+kWtSyuNCiMqAjrdoFx9CCJzefRfXTz8lbtcuQt4ZTlpcfJ7XNzc2Z0brGVSwrcDoPaO5FHVJtwHZlNYWedSYwG9vQHSwbtvTA41GMKJlFVaOaIIQ0HP+EWbvuarKqyhKMZHrDYnFga43JOZVzN9/c2fiJ5h7eFB+wXyMS5XK87rh8eH029yPNJnGUr+llLMpl/tKOW7wHPzaWVs9eMg27VFKERDzKIVP1p5jU0AYzao68lNPH1xszQt7WIqiZENvNyQKIcoJIdYKIe4KISKEEKuFEDp+yhVPdl27Um7mTJIuX+ZW//6kROS9Sm9pq9LMbzef5LRkRuwcQVRilG6DKe2lnTOJCdUWeUwqGldO2VmYMKt3Xb7t7sXJWw/wm36AvZfzX2FZUZSiIy+ntn5FWxalDNp+IhsyHlOyYdPal/ILF5J6J4xbffqSHJz3U0tV7Kswu81swuPDeX/n+ySk6HiHeIUm0PM37dHJij6QokMveT0SQvBWQ3c2jHwVJ2szBv16nK83B5Kcmr8r3xRFKRryEiTOUspfpZSpGT+LgfzXZS9BrBo1xH3JEtLj47nZty+Jl/M+6e3j4sPUFlO5GHWRcXvHkZKu41VO1Tto7zO5sR9WD4U0He+m16NqrjasH9mMvo3cWbD/Om/OO0xwpCqvoigvm7wEyX0hRD8hhFHGTz8g0tADe9lZeHlSYfkyhMaIW/0HkHD6dJ7X9XX35b+N/8uhO4eYfGgy6VLHb+p13oKO38KljbBxbKHV5cqOuYkRX3XzYm7fety4H0+nGQdYf+Z2YQ9LUZR8yEuQDEF76W84EAb0QFs2RcmFWZUqVFi+HKNS9gQPGUrcoUN5Xte/uj8jfUay4foGpp2cpvtgGr+rrc11eins/Fz37emZn5cbm8c0p0ZpG8asOMNHq86SkFx0jp4URXm+vJRICZZSdpVSOkspXaSUb6C9OVHJA9NyZam4bBmm7u6EjHiX2G3b87zuO97v8FaNt/j1wq/8diH/db2e4TsJ6g+BQ9Pg0HTdt6dn5UpZ8uc7jRnpW5WVJ0N5beZBAsNiC3tYiqLk4kUbR4zT6yiKOWNnZyos/Q0LT09uf/AB0atX52k9IQQTGk6gXYV2TD0xlU3XN+k2ECGg0/dQuxvs+C+cLnq1N42NNHzYoQbLhjYiNjGV12cfYumRm6q8iqIUYS8aJEKvoygBjGxtcf/lZ6yaNiVs0qdELsrbhW9GGiO+af4N9V3r8+mhTzl857BuA9EYQbf5UNkX/h4FgUWzWEGzqk5sGdOcplUc+Wz9BUYsO0l0QnJhD0tRlGy8aJCor4cvQGNpSfk5s7Hp2JG7//sfd6dNy9M3bTMjM2a0nkFlu8p8sOcDLkRe0G0gxmbQaxmUqQurhsCNA7ptz0CcrM1YNLABkzrVYvelu3SafoDjN3W8v0ZRFL17bpAIIR4KIWKz+XmI9p4S5QUIU1PK/vA99m/2IHLefCL+7//yVDnYxtSGuW3nYm9mz3s73yM4VsfSJ2bW0HcVlKoIf/SGO2d0256BaDSCt1tUZtWIphgbaeg1/wgzdgWRpsqrKEqR8dwgkVLaSClts/mxkVIaF+QgixthZETpL77AYegQHvz+B3c++hiZkvv9Ii6WLsxrN490mc7wHcO5/+i+bgOxdID+a7XFHpf5w/2rum3PgOqUt2fT6Fd5rU4Zftxxhb4//0N4TNG4wVJRSroXPbWl6EgIgev48TiPG0fsxo2EjhxFemLuH4yV7Coxu81sIhMjeW/ne8Qlx+k2ELuy0D+jGdbSNyCm6N7DYWNuwrRePkzt4c3ZkBj8pu9n96W8l6FRFMUwVJAUMqd33qb055OJ27+fkGFvkxaXezB4O3vzQ8sfCHoQxNi9Y0lO03ES2qkq9FsFj6JhWXdIKLrzEEII3qxfng2jXsXV1pwhi0/wxYaLJKWqlr6KUlhUkBQBpd56izLfTyXhzBmCBwwkNSr3D/Lm5ZozpdkUjoYdZdLBSbrf/V6mLvT+HaKuw0LfQu+ymJuqLtase78ZA5tUYNGhG/jPPcyN+3kv368oiv6oICki7Dp3pvzsWSRdu8atfv1JCQvLdZ2uVbrywSsfsPXmVqYen6r7vRaVWsCA9SCMYGk3WDMc4otuNRxzEyOmvO7J/P6vEBL1iC4zDrD2dGhhD0tRSpy8lJHP7uqtkIzS8pULYpAlhXXLlrj/8jOpd+9ys29fkm7cyHWdwbUH069WP5YFLmPR+UW6D6JCU3j3sLacyvlVMKs+nF1RpOpzPa1D7dJsGdOc2mXs+ODPs4z76wzxSaq8iqIUlLwckfwIjEdbQr4c8CGwEFgB6OGTS8nKsn59Kvy2BJmYxK1+/UkMDMxxeSEE4xuMx6+iH9NOTWP91fW6D8LEHFp/CsMPgGMVWDtcOxEfdV33bRtIGXsLfn+7EaPbVGPt6du8NvMg52/HFPawFKVEyLVDohDiqJSy0VOP/SOlbCyEOCulrGPQEepBQXVI1Kek6zcIHjqU9IcPKT9/HpavvJLj8slpyby36z1OhJ9gZuuZNC/XXD8DSU+HE7/AzimQngKtJkCTkWBkop/tG8CRa5GM/fM0D+JTmNipJoOaVkQIVYxBUfJLbx0SgXQhRE8hhCbjp2eW54ru+Y6XnFnlSlRcvgxjJyeChw4jbv/+HJc3NTJlWqtpVC9Vnf/s+w8B9wL0MxCNBhq+DSOPQdW22srBC1pB6En9bN8AmlRxZMuYFjSv5sSUDRd5+7cTRMWr8iqKYih5CZK+QH/gbsZPf6CfEMICGGnAsZV4JmXKUGH5MkwrVyLkvfeJ3bw5x+WtTa2Z03YOjuaOvL/rfW7E5D7Hkme2ZeCt5drSKgmR8HMb2PJxkWnh+zQHK1N+Hlif/3bxYP+V+3SafoB/rhfdCwcU5WWW66mt4uBlPLWVVdrDh4S8+y6PTp6i9OefU6pXzxyXD44Npv+W/pgbmbO001JcLF30O6DEWNj1BRz/WRswnX+AGn763Ycenb8dw6g/TnMrMp5RrasxqnVVjI3UBYuKkhu9ndoSQpTLuELrrhAiQgixWghRTj/DVPLCyMYG94ULsWrRnPDJk7m/cGGOy7vbujOn7Ryik6J5d+e7PEzW81GDuS10/h6GbgczW/jjLfhrADwM1+9+9MSzrB0bRr3KGz5lmb4riD4Lj3In+lFhD0tRio28fC37FfgbbaHGssCGjMeUAqSxsKD8rFnYdu7MvR9+5O4PP+R430htx9r85PsT12OuM3r3aJLSkvQ/qPINYfh+aP0ZXN4KsxrCiUXaCfoixtrMmB97+fDDm3U4fyeGTjMOsOOiKq+iKPqQlyBxllL+KqVMzfhZDDgbeFxKNoSJCWWm/g/73m8RufBnwid/jkx7fmmQpmWa8lWzrzgRcYKJByaSlm6AMiLGptDiQ3jvCLh5w8YP4Fc/uHtJ//vSA/9XyrFx1KuUtbfg7d9O8PnfF0hMUeVVFEUXeQmS+0KIfkIIo4yffoCatSwkQqOh9H//i+Pw4UT/9Re3P/wQmfz8K5I6Ve7E+Prj2XFrB98c+8ZwnQYdq8DADfD6bLh3Cea9Cnu+hlQDHAnpqLKzNWvea8qQZpVYfPgm3eYc5vDV+6SmFb0jKUV5GeTlPhJ3YBbQBO3lvoeB0VJKHRtiFJyXfbL9eSJ/WcTdqVOxat6ccjOmo7GweO6yP574kV8v/MpIn5EMrzPcsAOLuwfbJsK5leBYDV6bDhWbGXafL2hXYATjVwUQFZ+Mg5Up7T1c8fNyo0llR0yN1YS8UrLldbL9ha7aEkKMlVJOy8NyHYHpgBHws5Ty26eeHwcMA1KBe8AQKeWtjOe2Ao2Bg1LKLlnWqYT2rnoH4BTQX0qZ400CxTVIAKJXrSLsv5Ox8PGh/Ly5GNnaZrtcukzn04OfsuH6Bj5v8jn+1f0NP7ignbDpA4gOhnoDod0UsChl+P3mU0JyKnsv32PL+XB2B0YQn5yGrbkxbT1c6eTpxqvVnDA3MSrsYSpKgTN0kARLKd1zWcYIuAK0A0KB40BvKeXFLMv4AkellAlCiHeBVlLKXhnPtQEsgeFPBclfwBop5QohxDzgrJRybk5jKc5BAhC7dRu3x4/HrGpV3BcuwNjJKdvlUtJTGLV7FEfuHGFaq2n4uvsafnDJ8bD3GzgyBywdwe87qN0Niuid5okpaRwIus+W82HsvBhBbGIqVqZGtK7lSifP0rSs4YylqerrppQMhg6SECll+VyWaQJ8LqXskPH3RAAp5TfPWb4uMEtK2SzLY62ADx8HidDWubgHlJZSpj69j+cp7kECEHfgIKGjR2Pi4oL7ol8wKVs22+USUhIYum0oQdFBLGy/kLoudQtmgGFn4e/REHYGqnXQXj5sn+N3kUKXnJrO4Wv32Xo+nG0XwnmQkIK5iYZW1V3w8ypN65ou2JgX3VIxiqKronBE0gPoKKUclvF3f6CRlDLbu+GFELOAcCnll1kea8WTQeIE/COlrJrxd3lgi5TSM5vtvQO8A+Du7v7KrVu38v06XzYJp04TMmIEGktL3Bf9glnl7IszRyVGMWDLAB4kPmBJxyVULVW1YAaYlgrH5sPuLwGhLQzZaDhoiv5po9S0dI7diGLL+XC2Xgjn3sMkTI00NK/mhJ+XG+1quWJnqUJFKV50DhIhxEOyr6UlAIvc+rYLId4EOjwVJA2llKOyWbYf2nIrLaWUSVkeb8WTQeIMHHkqSDZLKb1yGktJOCJ5LPHyZYKHDoO0NMovXIiFZ+1slwt9GEr/Lf0xEkYs67SM0lalC26QD27Bpv/A1R3ahlqvzdBeOvySSE+XnAx+wJZz4Ww9H8admESMNYImVRzp5OVGew9XHK3NCnuYiqIzgx6R5HEAeTq1JYRoC8xEGyJ3n3quFerUVr4l37pF8OAhpMXEUG7uHKwaNsx2uctRlxm0dRCulq4s8VuCnZldwQ1SSriwRluvKyEKmrwPrSaCqWXBjUEPpJScDY1hy/kwtpwLJzgqAY2AhpUc6OTlRofapXG1NS/sYSrKCykKQWKMdrK9DXAb7WR7HynlhSzL1AVWoT0FFpTNNlqRJUgyHlsJrM4y2R4gpZyT01hKWpAApEREEDxkKCmhoZSd9hM2vtlPrB8LO8aInSPwdPJkQbsFmBsX8IdeQhTsnAynfgP7CtDlJ6japmDHoCdSSi6GxbL1fDibz4Vx7V48QkA991L4eZamo2dpypV6uYJSKdkKPUgyBtEJmIb28t9FUsqvhBBfACeklH8LIXYCXsDjvrLBUsquGeseAGoC1mhvgBwqpdyW0ZXx8eW/p4F+WU+HZackBglA6oMHhLwznMSLFynz7TfYvfZatsttu7mN8fvG07J8S35q9RPGmkK4KunmQdgwBiKvgldP6PA1WL/cBRSCIh6yJSNULoVr653VKWdHR083/DxLU9HJqpBHqCg5KxJBUlSU1CABSIuLJ/T990k4dgzXzz7FoU+fbJf7PfB3vjn2Df7V/JncZHLhNIJKSYQDP8DBn8DMGtp/BT59iuylwvlx8348W86Hs+V8GAGh2s6Ntdxs8fMsjZ9naaq52hTyCBXlWSpIsijJQQKQnpTE7Q/GEbd7N85jx+A4fHi2QTHj1AwWnltI1ypdGVV3VMFOwGd195L26CTkH6jUArpM05ZgKSZCHySw9Xw4W86Hc/LWAwCqulhnnv7ycLNVHR2VIkEFSRYlPUgAZGoqYZMmEbP+bxwGD8blo/HPfFhJKZl2ahq/XfgNIQT+1fwZ5jUMVyvXgh9wejqc/FXbkTEtGVp+BE1HF+kWvy8iIjaRbRe0p7+O3YgiXUIFR0s6epbGz9ONOuXsVKgohUYFSRYqSLRkejoRX3/Dg2XLsPPvjtuUKQjjZ+dDbsfdZmHAQtZfXY8Qgh7VezDUc2jhBEpsGGz5CAL/BhcP7aXC5RsU/DgKwP24JHZcjGDzuTCOXIskNV1Sxs5cO6fiVZpX3Euh0ahQUQqOCpIsVJD8S0rJ/VmzuT97Njbt2lHmh+/RmJpmu2zWQNEIjTZQvIbqv+NiXlzaBJs+hIdh2h7yrT/TNtgqpqITktkZeJct58I4EHSf5LR0XGzM6FBbO6fSsJKD6vKoGJwKkixUkDwr6rffiPj6G6yaNqXczBlorJ5/BVHow1AWntMGipEw4s0abzLUcyjOlgV8VVVirPau+GMLwMZNW2alZueCHUMheJiYwu5Ld9lyLpy9V+6SmJKeWam4o2dpmlZxUpWKFYNQQZKFCpLsRa9ZS9inn2Lh5UX5BfMxssv5hsSQhyEsDFjI39f+xlhjzJvV32SI55CCD5TQE9q6XXcvQK3XwG8q2LoV7BgKSUJyKvsu32NzNpWK/TzdaK4qFSt6pIIkCxUkz/dw505ufzAO00qVKP/zQkxccj9tFfIwhAUBC9hwbUNmoAz1GoqTRfZVhw0iLQUOz4R934GRKbSdDK8MAU3J+WaemJLGwaD7bM6mUrGfZ2laqUrFio5UkGShgiRn8UeOEPL+SIydnHBf9Aum5crlab2Q2BDmB8xn4/WNGGuM6VmjJ0M8hxRsoERe07b3vbEPyjfSNtFyqVVw+y8iklPTOXI9ki3nwth+MYKo+GRVqVjRmQqSLFSQ5O7R2bMEvzMcjakp5WbOwMLHJ8/rBscGZwaKqcaUnjV6MthzcMEFipRwdgVs+wSSHsKrY6H5h2BSMmtc5VSpuKNnadp5uGJvmf0FFoqSlQqSLFSQ5E3ilSuEDB1G6r17mNWqhb2/P3ZdOmNkb5+n9W/F3mJBwILMQOlVoxeDPQfjaOFo4JFniL8P2yZBwApwqKI9OqnUvGD2XUSlp0tOBT9gczaVijtnFJUsZaVCRcmeCpIsVJDkXVpsLDEbNxKzajWJFy8iTEywadcWu+7+WDVpjDDKfSL3Vuwt5p+dz6YbmzAzMqNXjV4Mqj2o4ALl2m7t6a4HN6Fuf2j3BVg6FMy+i7DsKhUbawTNqjrR2duNDh6lVU8V5QkqSLJQQfJiEgMDiV6zlti//yYtJgbjMm7Yv9ENu+7d8jSPciPmBgsCFrD5xmbMjMx4q8ZbDPIchIN5AXyoJydoJ+IPz9SGSMdvwdO/WNTt0gcpJedvx7Lx3B02BYQR+uARJkaC5tWc6ezlRrvartiqOZUSTwVJFipIdJOenEzcrl1Er15D/KFDICWWjRtj7++PTbu2aMxznou4HnNdGyjXN2NubM5bNd9iUO0CCpSwANgwGu6chqrtoPMPUKqC4ff7Enl8pLIpQBsqd2ISMTXS0KK6M1283Wjr4Yq1mbr6qyRSQZKFChL9SQkLI2bdOqJXryElNBSNjQ22XTpj390fc8/aOdaFuh5znfln57PlxhbMjc3pXbM3g2oPopR5KcMOOj1NexPjrv8DJPhOgkYjwEh9OD4tPV1yJjSajWfD2HwujPDYREyNNfjWcKazdxna1HTBSoVKiaGCJAsVJPon09NJOHac6DWrebhtOzIpCbMaNbD3747ta69hXOr54XA9+jrzAuax9cZWzI3N6VOzDwNrDzR8oESHaFv8Bm0Dtzraul1l8n51WknzeKJ+Y0AYm86Fce9hEuYmGlrXdKGzVxla13TBwlTd/FicqSDJQgWJYaXFxhK7eTPRq9eQeO4cmJhg07o19v7dsWrW7LkT9NeirzH/7Hy23tyKhbEFfWr1YaDHQOzN83aV2AuREi6ug80fQcJ9aPwe+H4CpqrJVE7S0iUnbkaxMSCMLefDuB+XjIWJEW1qudDF241WNVzUHfXFkAqSLFSQFJzEy1eIWbOGmL//Ju3BA4xdXbHr9gb23btj6u6e7TpXH1xlfsB8tt3choWxBX1r9WVg7YGG7SH/6IG2RP3JxWDnrr33pGZnsCmkHiwvkbR0ydEbkWwMCGPr+XCi4pOxMjWirYcrnb3caFHdWYVKMaGCJAsVJAVPJifzcM9eotesJv7AQUhPx7JBA+z8u2PboQMaC4tn1gl6EJQZKFYmVpmnvAwaKLcOa6sK370ACCjXAGp1gZpdilUzLUNJTUvnn+tRbAy4w9YL4UQnpGBjZkw7D1c6e7vxajUnzIxVqLysVJBkoYKkcKVERBCzdh3Ra9aQEhyMxsoK286dsffvjrm39zMT9EEPgph3dh7bb23HysSKvrX6MsBjgOECRUq4GwiXNkLgBggP0D7u4qENlFpdoLS3unQ4Fylp6Ry+FsnGs3fYdiGc2MRUbMyN6VC7NJ293WimqhS/dFSQZKGCpGiQUvLoxAmiV68hdts25KNHmFWril13f+y6voax45M3LF55cIV5Z+ex49YOrE2s6VurL/09+hv2CAXgwS1t/5NLGyH4CMh07emvWl20p7/cm4BGfcvOSXJqOoeu3mdDwB12XIjgYVIqdhYmdMwIlaZVHFU/lZeACpIsVJAUPWlxccRu3kzM6jU8OnsWjI2x8W2Fnb8/1q+++kTnxstRl5kfMD8zUPp59KO/R39sTQugsVX8fbi8WRss1/ZAWhJYOkINP6j5GlRuVWJreuVVUmoaB67cZ2PAHXZc1Ja+L2VpQkdPN7p4u9FINekqslSQZKGCpGhLunqV6DVriVm/nrTISIydnbF74w3sunfDrFKlzOUuR11m3tl57AzeiY2JDf08+tHPo1/BBApoC0Je3QmBGyFoOyTFgqk1VG2r7YtSrR2YG/ho6SWXmJLGviv32BQQxs7ACBKS03CyNqWjZ2k6e5WhYSUHjFQ74SJDBUkWKkheDjIlhbh9+4hevYa4/fshLQ2LV17Bvnt3bDt2yOzieCnqEvPOzmNX8C5sTGzo79Gffh79sDG1KbjBpibBjQNwaQNc2gzxd0FjApVbaudVanYG60JoSfwSeZScxt7Ld9l4LozdgXd5lJKGs40ZnTxL09m7DPUrqB71hU0FSRYqSF4+KXfvErN+PTGr15B88yYaS0tsOvlh390fi7o+CCEIjAxk3tl57A7ZjY1pRqDUKuBAAe2d86HHtRP1lzZqi0UitP1RHl8B5lApt62UaAnJqey+dJdNAWHsvnSXpNR0XG3N6OSlPf1Vt7wKlcKggiQLFSQvLyklj06fJnr1amK3bEUmJGBauTL2/t2x69oVY2dnAiMDmXt2LntC9mBjasMAjwH0q9UPa1PrwhgwRFzIuAJsI0Sc0z7u6vnvFWCunuoKsBzEJaWyKzCCTQFh7L1yj+TUdMrYmdPJy43O3m74lLfPsRSPoj3ai4xPIio+mRqlbV74EmwVJFmoICke0uLiebhtK9Gr1/Do1CkwMsK6ZUvs/btj3aIFgbFBzD07l70he7E1tWWAxwD61upbOIHy2IOb2kC5tBGC/wEk2FfQzqnU7ALlG6orwHLwMDGFnRmhsu/KPVLSJGXtLejirQ0Vr7J2JSJUElPSiIxPJjIuicj4ZKLikomMz/p7cubzUfHJJCSnZa676z8tqeL8Yv8NqCDJQgVJ8ZN0/QYxa1YTvW49affvY+TkhN3rXbH39+eq3SPmnZnH3lBtoAysPZA+NfsUbqAAxN2Fy1u0oXJ9L6Qlg5VzlivAWoKxWeGOsQiLeZTCjosRbAq4w4Gg+6SmS8o7WNDZqwxdvN2oXcb2pQmVxJQ0ouKTiXwcCHHJ2r+zhMH9+GSi4pOIiksmPkswZGVqrMHRyhRHa1McrMy0v1uZ4mBtipOVGQ5WpjSq7PDCbZZVkGShgqT4kikpxB04SPSa1cTt3QepqVj4+GDfw5/bDSsyN2gx+0L3YWdmx0CPgfSp1QcrkyJQVysxFq7uyLgCbAckPwRTG+2VX7W6QLX2YFbAcz0vkeiEZLZfiGDjuTAOXb1PWrqkoqMlnb3d6OJdhpqlbQo0VB4HwzNhEOuPn5kAABn2SURBVJcRBpm/a597bjAYaXDIDAbTjJDQBoLT47CwzggLK1OszYwN+jpVkGShgqRkSL1/n5j1fxO9ejXJ168jLCyw7diRqLZ1mSv3sP/2AezM7BhUexC9a/YuGoEC2ivAru/79wqwhPtgZKq9R6VmF6jRCaydC3uURVZUfDLbLoSzKSCMw9fuky6hsrMVXbzc6FKnDNVd8x/ISalZjxi0YZD5e9bTShnLxCWlZrsdEyOREQhZA+DJMHC0Nss8irAxcDDklwqSLFSQlCxSShLPntXeQb95M+nx8ZhWqMCjjs1YUv4mW+OPYW9mn3nKy9LEsrCH/K/0NAg5mjGvsgGig0FooHzjf68AU425nut+XBJbz2tD5Z8bkUgJ1Vys6eztRjsPV6Qk88jhfsZRwxNhkREUD58TDMYa8cSH/+MjB6eMo4anjxyKWjDklwqSLFSQlFzpCQnEbttOzOrVJJw4ARoN6Y182FI7iWUOl7CxLMWg2oPoUb2H4Uuv5JeUEH5OO6dyaRNEnNc+XtpLO6dSq4u2HthL/EFlSHcfJrL1fDgbA8I4fjOK7D7qHgfD49NJjlZPnkb693dteNiav9zBkF8qSLJQQaIAJN/8//buPTrK+77z+PurGWkkBBLiIgzCXESMj4kxiJvBErZ84tqOnWCDvU2cza5z2tTbnCY9Z1Pn7LbNttkkvewme7ptk9M2ad2kzdnk7NoCYyeOU9s41giMEUJcbQPmYhAYiYsEGt3m8t0/fo9Gj4SEBqTRjND3dc4czTzzPDO/HyPNh9/zPL/vc8LNoN+yhVhzMzq1iD0rivhJ+VnOlAZYOmMplWWVrC9bz5LpS8iRLCvbcfGYC5R3X3ajFhRKFnojlU+7ysU5WdbmLHHuchfbPzhPQW6wbxRRGKKoYGIFw/WyIPGxIDF+GosRqauj9YUarmzbBtEobYtmEV6Wy/PzPyKSDyWhEu4pu4fKOZVUllWOzfXlr8eVc14NsJfd8ZVEFCbPcsdT7vgULLgXgnmZbqUZ5yxIfCxIzFBiFy/StnUrbS/U0H3kCITyuLLu42xfUcDmyYe5GG1FED4+/eNUllVSVVbF0hlLCWTT3I+uNnfm17svuZ/RCISK3Jlfd3wKPvYbEMrwqc9mXMqKIBGRh4G/BgLAP6rqXw54/qvAF4EY0AL8lqqe9J57Gvi6t+q3VfXH3vI3gdlAp/fcg6rafK12WJCY4agqXQcO0ra5hraXf07i8mWCc+bQ81Al76yYzOvRfew/v5+EJijKK+KeOfdQVVZFZVklMwpmZLr5faJdbo7Key+5OSsdFyAQgkX3950BVjh92JcxBrIgSEQkABwGfgM4DewCnlLVQ7517gd2qmqHiHwJqFbVz4jINKAeWAUosBtYqaqXvCB5VlVTTgYLEnM9Et3dXHntNdpeqCGyYweoMmntWvI+/RD7lkzirQs7qWuq40LXBQDumHZHMlSWzVxGMCc4zDuMkXgMTr3dd1ylzTsDbN49fSOV6YvsYL0ZUjYEyTrgG6r6kPf4DwFU9S+GWL8C+J6qVorIU7hQ+U/ec/8AvKmqP7UgMWMpeuYMrVu20LZ5C9FTp8iZPJmiRx6haOPjfHhriLqz26k9Xcvelr3ENc6U3CmsnbPWBcucSmYVzsp0FxxVd+XH3nItzd7/54pvdfNVem+FWTS6MhmXDUHyJPCwqn7Re/wfgLtV9ctDrP894CNV/baIPAvkq+q3vef+G9Cpqt/1gmQ6EAdewO32umYnLEjMSGkiQceuetpqarj8q1+hnZ3kLVrE1E0bKd6wgY7iEDvP7iTcFCbcFKa5w+1tva3kNqrKqqiaU0VFaQW5gRsrVTHqLh5zF+o6tg2Ov+WOs4C7pHB5tdsVNm8d5BZkspUmw7IhSP4d8NCAIFmjql8ZZN3PA18G7lPVbhH5GhAaECQdqvq/RKRMVZtEZAouSH6iqv8yyGs+AzwDMG/evJUnT55MSz/NxBNvb+fyK6/QVrOZzj17XPHIe+91xSPvuw+CQY60HqGuqY5wU5iG5gZiiRiTgpO4e/bdLljKqpgzeU6mu+Ik4nCmEY694c4A+/BtdxZYIATz1rpQKb/fhYydXjyhZEOQpLRrS0QeAP4WFyLN3rIhd20N2PYLwKqhRjm9bERi0qX72DHaampoffFF4i3nCUybRvGGDRRv2kj+4sUARKKRfqOVs5GzAJQXlyePrayatYq8QJacrtsTgZPb3UH7D7ZB80G3vGCaKyxZXu2CxWbY3/SyIUiCuIPtnwCacAfbP6eqB33rVADP43aBHfEtn4Y7wL7CW9QArAQuA1NV9byI5AI/BV5T1b+/VlssSEy6aSxGe20tbTWbufLmmxCNkr90KVM3baTo0UcJFLnLAasqx9uOJ0Ol/lw90USUgmABq29ZndwNdmvRrZntkN+Vc3D81327wq64IGRaeV+oLFwPBSWZbKVJg4wHideIR4D/jTv99zlV/TMR+SZQr6pbReQ1YCng/Wbyoapu8Lb9LeCPvOV/pqr/LCKFwFtArvearwFfVdXBS2l6LEjMWIpdvMjll16i9YUaug8fRkIhpjzwAMWbNlK4bh3i2z3UEe2g/lw9tadrCTeFOd1+GoD5RfOTB+xX37Ka/GB+prrTnyqcP+yFyptwohZ62t3ZYHMqXKgsuh/mrrEJkTeBrAiSbGFBYjJBVek6eIi2mhraXn7Zm5sym6mPb6R400by5s69apuTl08mRyu7PtpFd7ybUCDEqlmrksdW5hfNz56yHvEonK53oXJsm7uvccidBPMrveMr1VYTbJyyIPGxIDGZlujupv3112l9oYbI9u1ubsqaNUx9YhNTHnyQnIKrz47qinWx+9zuZLCcuHwCgLLJZclQWXPLmuyqXtzVBifqXKh8sA0ueHusC0v7zgYrr4aiLDnRwFyTBYmPBYnJJtGzZ2l78UVaazYT/fBDcgoLKXrkEYo3baRg+fIhRxunrpyirqmOuqY6dn60k85YJ7k5uayYtYL1ZeupKquivLg8e0YrAG2nvdGKd4u0uOUzbu87G2xBpV3EK0tZkPhYkJhspKp01te766a8+qqbm1Je7g7Qb9hAbmnpkNv2xHtoaG4gfDpM3Zk6jrYeBWB24exkTbC7b7k785cX9ksk3BlgvcdXTm6HWCfkBF3l4nJvtFK2EgJZUh1ggrMg8bEgMdku3h7hyi9fobVmM50NDW5uyvr1FD+xiSn33YfkXfvA9dn2s4TPhKlrquPts28TiUYISpCKWRVUznHBsrhkcXaNVqJdcPqdvrPBzjQC6gpOLljftyts+sfs+EqGWJD4WJCY8aT72HHaNnvXTWlpIVBS4s1N2UT+7YuH3T4aj9LY0ki4yQXL+5feB6C0oDQ5Wlk7Zy1FeUXp7sr16bjoZtkf80Ysl0645UVz+0Jl4X122eExZEHiY0FixqPBrpuSf+edFG/aSPGjjxIoTu2Kjs0dzclZ9jvO7OBK9AoBCXD7tNupKK1I3konDb0rLSMuHu8LlWO/hq5Wt3zWUlhU7cJl3j2Ql0UnG9xkLEh8LEjMeBe7dMnNTanZTPd77yF5ed7clE0UrluLBFK7PkosEWP/+f3UNdXR0NzA/pb9dMW7AHc2mD9YFk1dlD1XiUzE4Wxj32z7Uzsh3gOBPFfGpbzaHWOZvQyy6Vox45wFiY8FibmZdB06ROsL3tyUtjaCs2dT/PhjTN24kbx5867rtaKJKO9deI89zXuSt97y+FPyprB85vJksNw5487smRjZ0wEfbveOr/wazu13ywtKYOG9fQfupy3MZCuHpurm4MS7Idbj/ex24RjrvT9gWfLngG2GfM77+em/geKyG2qmBYmPBYm5GSW6u2l/4w1aazYTCYfd3JTVqyl+YhNFDz5IzqTr3+Wjqpy6cqpfsBxrOwZAMCfIkulLqJjpgmV56XKmF2TJRbLam12g9E6MvNzklpcs6AuVBevdbPvkF23X1V+6g31xx7oG+TK/wS91/zqM0ndvThCC+W50Fgz5foZcf598zv073AALEh8LEnOzi370EW1bXqR1cw3Rk71zUz5J8cZNFFQMPTclFa1drTS2NCaD5cD5A0QTUcCVcukdsSwvXc7CooWZPzNMFc4f6QuV47XQc2X0Xj8nd8AX9oAv7kDIPR7yuYHrDPFcIM8FxFXLBrxuGisyW5D4WJCYiUJV6dy9u29uSkcHeQsXugP0jz12zbkpqeqOd3PowqFksDQ2N9La7Q6El4RKWFa6jBWlK6gorWDJ9CWZr2ocj0JTgzvVGK7jS33sv7izjQWJjwWJmYji7RGuvPpLNzdl9243N6WqiuJNm5hyf/Wwc1NSpaocv3ycPee8YGlp5ORld/2fvJw87pxxZ79RS3EotbPNTOZZkPhYkJiJrvv4cdo2b3FzU5qbCZSUMLm6msKqSgrvuYdgyeiWgD/feZ69zXtpaG6gsbmRQxcOEdMYAIuKF7G8dDkrZq2gYmYFc6fMzfzuMDMoCxIfCxJjHI3HidTV0fbiViLhMPG2NhAhf+lSJldVUVhVRcFdS5Hg6JYo6Yx1cuD8geTusL3Ne7kSdcctZhTMcKOVmS5cbp92O7k5WXJJ4gnOgsTHgsSYq2k8TteBA7SHw0Rqw3Tu2weJBDlFRRSuW0dhVSWTq6rInT171N87oQmOth6lsbkxOWppandnWhUEC1g6YynLS92px8tmLmNKnhV1zAQLEh8LEmOGF29rI7JjRzJYYufOAZD3sUVMrlpPYVUVk1avIicUSsv7n4ucY0/LnuSxlvcvvU9CEwjCbSW3JY+zrChdwezJox9u5moWJD4WJMZcH1Wl5+hR2mvDRMJhOurr0Z4eJBRi0po1TK6qpHD9evIWpu9030g0wr6WfclRy76WfXTEOgCYNWlWv1n4i0sWE7AZ7aPOgsTHgsSYkUl0dtKxa1dytNJz/DgAwTmzvdFKJYXr1hGYkr5dULFEjCOXjiR3hTU0N9Dc0QxAYW4hd824ywXLrArumnFXdl3wa5yyIPGxIDFmdPWcbiISDhOpCxPZvoNEJAKBAAXLl7vRStV68j++pN/16UebqnI2crZfsBy9dBRF+xWlXF66nBWlK7KvKOU4YEHiY0FiTPpoNErn3r3J3WBdBw8CECgpobCy0h20r6wkODP95d8v91xmX8s+Gs410NjS2K8o5S2Ft1ASKiE/mE9+IJ9QMERBoMA97r0F+n4WBAv6rxssSD5fECwgFAiRH8wnFAhlT3HLUWZB4mNBYszYiV24QGT7dtpra4nUbSd+wRWBDN1xR/IU40kVy0dtQuS19BalbGhu4NCFQ3REO+iMd9IV63K3eP+f3fHuG3qf3oAJBUL9AuhaAdUbRP6A6rdNMD8ZdL3rjnVgWZD4WJAYkxmaSND17rtEwnVEamvpaGyEWIycSZOYtHYtk9e7YMm79dZMNxVwpyX3BkpXrGvw0PHud8Y6k+t1xdzjrngX3bHua4ZV7/0b0RsogwVPQaCAUDDUb9SUH8zns7d/9oaLa1qQ+FiQGJMd4u3tdLz9dvKgfbTJzR3JnT+v76D9mjXkFBZmuKXpldAEPfGefqHUG2C99wcLoN6A6rfeNcKqK9bF1se3sqB4wQ2104LEx4LEmOyjqvScOEEkXEd7uJaOne+gXV1Ibi4FK1cmRyuhxVl2rflxpPf7/Ub//SxIfCxIjMl+ie5uOnfvpt3bDdZ95AgAwZkzKayqSltdMDM0CxIfCxJjxp/oRx8Rqatzu8G27yAxRnXBTB8LEh8LEmPGN43H6dq/P3mKcef+/WNWF2wisyDxsSAx5uYSb211dcG8YIk1uxnuY1UXbKKwIPGxIDHm5qWqdB8+QiQcpj1cS2f9bjQaRfLzmbR6dd8pxmmsCzYSquouDzzgpgCJRP9lCjDIuoNtrwoKwenTbnj3nwWJjwWJMRNHoqODyDvvJOeu9Jx0V2sMlpa604qTX9TuixbV5Bd2v2X9vtS95QO/2Blk3WsFg39ZIjEm/x7lv/g5ofLyG9o21SCxo1TGmJtKzqRJTKmuZkp1NQA9p0+7Csa7G9BoFKT3dFhx118X6b9MfLccGWQ5IN5yyem3fLDXkBzv/rCv4V/mWz9nkGX+9xvmNYLTb2wy4vWwEYkxxphBpToiuTkrjRljjBkzFiTGGGNGxILEGGPMiKQ1SETkYRF5X0SOish/HeT5r4rIIRHZJyKvi8h833NPi8gR7/a0b/lKEdnvvebfSDaez2eMMRNI2oJERALA94FPAkuAp0RkyYDV9gCrVPUu4Hngf3rbTgP+FLgbWAP8qYj0Ftj5O+AZ4Dbv9nC6+mCMMWZ46RyRrAGOquoxVe0BfgY85l9BVbepaof38G1grnf/IeDfVPWiql4C/g14WERmA0WqukPd6Wb/Ajyexj4YY4wZRjqDpAw45Xt82ls2lN8GXhlm2zLv/rCvKSLPiEi9iNS3tLRcZ9ONMcakKp1BMtixi0EnrYjI54FVwHeG2Tbl11TVH6jqKlVdNXMMrhVtjDETVTpntp8G/NfPnAucGbiSiDwA/DFwn6p2+7atHrDtm97yuQOWX/WaA+3evfu8iJy8jrb7zQDO3+C245X1eWKwPt/8Rtrf+cOvksaZ7SISBA4DnwCagF3A51T1oG+dCtxB9odV9Yhv+TRgN7DCW9QArFTViyKyC/gKsBP4BfC3qvqLtHTCtaU+lZmdNxPr88Rgfb75jVV/0zYiUdWYiHwZeBUIAM+p6kER+SZQr6pbcbuyJgP/zzuL90NV3eAFxrdw4QPwTVW96N3/EvAjoAB3TOUVjDHGZExaizZ6I4VfDFj2J777D1xj2+eA5wZZXg/cOYrNNMYYMwI2s314P8h0AzLA+jwxWJ9vfmPS3wlR/dcYY0z62IjEGGPMiFiQeFKoC/a7Xo2vRhEJD1LuZdwZrs++9Z4UERWRcX22Swqf8RdEpMX7jBtF5IuZaOdoSuUzFpHf9GreHRSR/zPWbRxtKXzOf+X7jA+LSGsm2jmaUujzPBHZJiJ7vNqGj4xqA9S7NOREvuHOKvsAKAfygL3AkgHrFPnubwB+mel2p7vP3npTgLdwJWxWZbrdaf6MvwB8L9NtHeM+34areVfiPS7NdLvT3ecB638Fd0Zpxtue5s/5B8CXvPtLgBOj2QYbkTip1AW77HtYyBAz6seRYfvs+RaumGbXWDYuDVLt780klT7/DvB9dTXtUNXmMW7jaLvez/kp4Kdj0rL0SaXPChR594tJYSL39bAgcVKqCyYivyciH+C+WH9/jNqWLsP22ZswequqvjyWDUuTVGu/PeEN/Z8XkVsHeX48SaXPi4HFIlInIm+LyHivpp1yjT/vshULgTfGoF3plEqfvwF8XkRO46ZkfGU0G2BB4qRUw0tVv6+qi4D/Anw97a1Kr2v2WURygL8C/mDMWpReqXzGLwEL1F3W4DXgx2lvVXql0ucgbvdWNe5/5/8oIlPT3K50SrkeH/BZ4HlVjaexPWMhlT4/BfxIVecCjwD/6v2NjwoLEielumA+P2P8l68frs9TcBM/3xSRE8BaYOs4PuA+7Gesqhe0r97bD4GVY9S2dEnl9/o08KKqRlX1OPA+LljGq+v5W/4s43+3FqTW598G/i+Aqu4A8nF1uEaFBYmzC7hNRBaKSB7uF2yrfwUR8f9xPQocYXy7Zp9VtU1VZ6jqAlVdgDvYvkFdZYHxKJXPeLbv4Qbg3TFsXzoM22dgC3A/gIjMwO3qOjamrRxdqfQZEbkdKAF2jHH70iGVPn+Iq3uIiNyBC5JRu75GWkukjBeaWl2wL3uViqPAJeDpoV8x+6XY55tGiv39fRHZAMSAi7izuMatFPv8KvCgiBwC4sDXVPVC5lo9Mtfxe/0U8DP1TmMaz1Ls8x8APxSR/4zb7fWF0ey7zWw3xhgzIrZryxhjzIhYkBhjjBkRCxJjjDEjYkFijDFmRCxIjDHGjIgFiTEjJCLfEJFns6AdJ7y5IMaMKQsSY4wxI2JBYswgRKRQRH4uIntF5ICIfMb/P34RWSUib/o2WSYib4jIERH5HW+d2SLylnfdiwMist5b/nciUu9d/+O/+97zhIj8uYjs8J5fISKvisgHIvK73jrV3mtu9q4h8veD1UwSkc+LyDvee/+DiATS+e9lJjYLEmMG9zBwRlWXqeqdwC+HWf8uXOmcdcCfiMgc4HPAq6q6HFgGNHrr/rGqrvK2uU9E7vK9zilVXQfUAj8CnsTVOfumb501uJnKS4FFwCZ/Q7wSGJ8BKr33jgP//jr6bsx1sRIpxgxuP/BdEfkfwMuqWisyWJHVpBdVtRPoFJFtuC/7XcBzIpILbFHV3iD5TRF5Bvf3Nxt3oaF93nO9JTz2A5NV9QpwRUS6fFV531HVYwAi8lOgCnje15ZP4ApO7vLaXACM9+uMmCxmQWLMIFT1sIisxJXc/gsR+RWuBlfvKD5/4CZXv4S+JSL34kYq/yoi38GNNJ4FVqvqJRH50YDX6q0+nPDd733c+/d61XsNeCzAj1X1D4fppjGjwnZtGTMIb9dUh6r+BPgusAI4QV9p+ScGbPKYiOSLyHTctT12eRdOalbVHwL/5L1GERAB2kRkFvDJG2jeGq/Saw5uF1Z4wPOvA0+KSKnXl2leW4xJCxuRGDO4pcB3RCSBq/j8Jdwuon8SkT8Cdg5Y/x3g58A84FuqekZEnga+JiJRoB34j6p6XET2AAdx5drrbqBtO4C/9Nr4FrDZ/6SqHhKRrwO/8sImCvwecPIG3suYYVn1X2PGERGpBp5V1U9lui3G9LJdW8YYY0bERiTGGGNGxEYkxhhjRsSCxBhjzIhYkBhjjBkRCxJjjDEjYkFijDFmRCxIjDHGjMj/B3M7UrVCqy6XAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x241994c94a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch3_1.best_score_, gsearch3_1.best_params_))\n",
    "test_means = gsearch3_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch3_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch3_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch3_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch3_1.cv_results_).to_csv('my_preds_subsampleh_colsample_bytree_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(colsample_bytree), len(subsample))\n",
    "train_scores = np.array(train_means).reshape(len(colsample_bytree), len(subsample))\n",
    "\n",
    "for i, value in enumerate(colsample_bytree):\n",
    "    pyplot.plot(subsample, -test_scores[i], label= 'test_colsample_bytree:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'subsample' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'subsample_vs_colsample_bytree1.png' )"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处使用GridSearchCV实现了对colsample_bytree和subsample参数的调整，并且得到了最佳参数为0.9和0.7，最佳结果为-0.200951。\n",
    "因为计算资源有限，所以暂时不进行对上述两个参数的进一步调整。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第四步：调整正则化参数：reg_alpha 和reg_lambda"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'reg_alpha': [1.5, 2], 'reg_lambda': [0.5, 1, 2]}"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#reg_alpha = [1e-3, 1e-2, 0.05, 0.1]    #default = 0\n",
    "#reg_lambda = [1e-3, 1e-2, 0.05, 0.1]   #default = 1\n",
    "\n",
    "reg_alpha = [ 1.5, 2]    #default = 0, 测试0.1,1，1.5，2\n",
    "reg_lambda = [0.5, 1, 2]      #default = 1，测试0.1， 0.5， 1，2\n",
    "\n",
    "param_test5_1 = dict(reg_alpha=reg_alpha, reg_lambda=reg_lambda)\n",
    "param_test5_1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.21615, std: 0.00442, params: {'reg_alpha': 1.5, 'reg_lambda': 0.5},\n",
       "  mean: -0.21849, std: 0.00515, params: {'reg_alpha': 1.5, 'reg_lambda': 1},\n",
       "  mean: -0.22188, std: 0.00448, params: {'reg_alpha': 1.5, 'reg_lambda': 2},\n",
       "  mean: -0.22720, std: 0.00428, params: {'reg_alpha': 2, 'reg_lambda': 0.5},\n",
       "  mean: -0.22873, std: 0.00433, params: {'reg_alpha': 2, 'reg_lambda': 1},\n",
       "  mean: -0.23229, std: 0.00448, params: {'reg_alpha': 2, 'reg_lambda': 2}],\n",
       " {'reg_alpha': 1.5, 'reg_lambda': 0.5},\n",
       " -0.21614622849929133)"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb5_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=999,  #第二轮参数调整得到的n_estimators最优值\n",
    "        max_depth=9,\n",
    "        min_child_weight=0,\n",
    "        gamma=0,\n",
    "        subsample=0.7,\n",
    "        colsample_bytree=0.9,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch5_1 = GridSearchCV(xgb5_1, param_grid = param_test5_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch5_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch5_1.grid_scores_, gsearch5_1.best_params_,     gsearch5_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([5638.06721997, 5483.25850058, 5463.67629681, 5399.54526114,\n",
       "        5431.71117148, 4779.35426421]),\n",
       " 'mean_score_time': array([69.79962745, 45.32693639, 45.30579176, 48.9241766 , 49.57772508,\n",
       "        39.42584367]),\n",
       " 'mean_test_score': array([-0.21614623, -0.21848715, -0.22188105, -0.22719556, -0.22873085,\n",
       "        -0.23229433]),\n",
       " 'mean_train_score': array([-0.05366833, -0.05582279, -0.05941334, -0.0648989 , -0.06657457,\n",
       "        -0.06983429]),\n",
       " 'param_reg_alpha': masked_array(data=[1.5, 1.5, 1.5, 2, 2, 2],\n",
       "              mask=[False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_reg_lambda': masked_array(data=[0.5, 1, 2, 0.5, 1, 2],\n",
       "              mask=[False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'reg_alpha': 1.5, 'reg_lambda': 0.5},\n",
       "  {'reg_alpha': 1.5, 'reg_lambda': 1},\n",
       "  {'reg_alpha': 1.5, 'reg_lambda': 2},\n",
       "  {'reg_alpha': 2, 'reg_lambda': 0.5},\n",
       "  {'reg_alpha': 2, 'reg_lambda': 1},\n",
       "  {'reg_alpha': 2, 'reg_lambda': 2}],\n",
       " 'rank_test_score': array([1, 2, 3, 4, 5, 6]),\n",
       " 'split0_test_score': array([-0.21560169, -0.21789216, -0.22111459, -0.22505342, -0.22768791,\n",
       "        -0.23120361]),\n",
       " 'split0_train_score': array([-0.05360065, -0.05620292, -0.05936964, -0.06481008, -0.06659251,\n",
       "        -0.07006115]),\n",
       " 'split1_test_score': array([-0.22419003, -0.22837672, -0.23004087, -0.23541156, -0.23648236,\n",
       "        -0.24048511]),\n",
       " 'split1_train_score': array([-0.05342209, -0.05555699, -0.05924021, -0.0645011 , -0.06605223,\n",
       "        -0.06933346]),\n",
       " 'split2_test_score': array([-0.21659571, -0.21717162, -0.22166829, -0.22712733, -0.22960242,\n",
       "        -0.23296205]),\n",
       " 'split2_train_score': array([-0.05363228, -0.05580562, -0.05942351, -0.0648625 , -0.06678736,\n",
       "        -0.07017613]),\n",
       " 'split3_test_score': array([-0.21283485, -0.21523846, -0.22020942, -0.22501569, -0.22621094,\n",
       "        -0.22921049]),\n",
       " 'split3_train_score': array([-0.05357273, -0.05566264, -0.05935643, -0.06483189, -0.06626349,\n",
       "        -0.06959518]),\n",
       " 'split4_test_score': array([-0.21150799, -0.21375582, -0.21637112, -0.22336911, -0.22366974,\n",
       "        -0.22760954]),\n",
       " 'split4_train_score': array([-0.0541139 , -0.05588577, -0.05967691, -0.06548895, -0.06717727,\n",
       "        -0.07000551]),\n",
       " 'std_fit_time': array([ 73.78838183,  26.79808001,  45.67079471,  14.84692108,\n",
       "         14.27403885, 802.67916527]),\n",
       " 'std_score_time': array([23.33770564,  3.16935445,  4.00127041,  6.26454554, 11.43459152,\n",
       "        11.37718763]),\n",
       " 'std_test_score': array([0.00441997, 0.00515423, 0.00448039, 0.00427766, 0.00433356,\n",
       "        0.00447623]),\n",
       " 'std_train_score': array([0.00023418, 0.00022138, 0.0001447 , 0.00032252, 0.00039458,\n",
       "        0.00031801])}"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch5_1.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.216146 using {'reg_alpha': 1.5, 'reg_lambda': 0.5}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAELCAYAAADz6wBxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VeW59//PRSBMIQkkYUqYR5Ew6BZRrGDVU7RVT1sfh9YBJ6zW06q/U9tT2x5/nJ6nTsenw7FV6lBre2p/2tZah3pO+4BWK0oQZZRBRNhhCgECIUCm6/fHWkk2MSQbkp29k3zfrxcv9l7r3ivXJuxcue973fdl7o6IiMiJ6pbsAEREpGNTIhERkVZRIhERkVZRIhERkVZRIhERkVZRIhERkVZRIhERkVZRIhERkVZRIhERkVbpnuwA2kNubq6PHDky2WGIiHQoy5Yt2+3ueS216xKJZOTIkRQVFSU7DBGRDsXMPo6nnYa2RESkVZRIRESkVZRIRESkVbrEHElTqqqqiEajHD58ONmhSCO9evWioKCAHj16JDsUEYlDl00k0WiUfv36MXLkSMws2eFIyN0pLS0lGo0yatSoZIcjInHoskNbhw8fJicnR0kkxZgZOTk56imKdCBdNpEASiIpSt8XkY6lSycSEZFOqbICPv47vPljqD6S8C+X0DkSM5sL/AhIAx5z93sbnb8TuBGoBkqA6939YzMbAfw+fF0P4Cfu/oiZ9QGeBcYANcCf3P1biXwPIiIprbYWdq+H4iKIFgV/71wDXhOcHz0bhkxNaAgJ65GYWRrwMHABMAm40swmNWq2HIi4+xTgOeD+8Ph24Ex3nwacDnzLzIaG5x5094nAdGCWmV2QqPfQmSxevJjPfe5zrW7TnA8++IAzzjiDnj178uCDDx6z3bx58xg1ahTTpk1j2rRpvPfeeyf8NUW6nPJd8MHL8NcF8NTFcN8I+Onp8MevwqrfQZ8cOOsOuPIZ+OeNCU8ikNgeyQxgo7tvAjCzZ4BLgDV1Ddx9UUz7JcBV4fHKmOM9CROeu1cAi+ramNm7QEEC30O7cXfcnW7dOu5o44ABA/jxj3/M888/32LbBx54gEsvvbQdohLpwKoOwfb3G3oa0WVQtiU4Z2kw6GQovBTyI1AQgZxxkISfIYlMJPnA1pjnUYLexbHcALxS98TMhgEvAWOBb7j7ttjGZpYNXEQwdNYq/++fVrNm2/7WXuYok4Zm8q8Xndxsm82bN3PBBRdwzjnn8NZbb3H77bfzyCOPcOTIEcaMGcOTTz5JRkYGL7/8MnfeeSe5ubmccsopbNq0iRdffLHJa77zzjvcfvvtHDp0iN69e/Pkk08yYcKEo9rcc889fPjhhxQXF7N161buuusubrrpJgDKy8u59NJLWbVqFaeeeiq/+tWvMDMWLFjAn/70Jw4dOsSZZ57Jo48++olJ8YEDBzJw4EBeeumlVvzLiXRRtbVQurHRENVqqK0OzmcNg/xT4fT5QeIYMhXS+yQ35lAiE0lTt954kw3NrgIiwOz6hu5bgSnhkNbzZvacu+8M23cHfgP8uK7H08Q15wPzAYYPH96a95FQ69at48knn2TBggV84Qtf4C9/+Qt9+/blvvvu46GHHuKuu+7i5ptv5vXXX2fUqFFceeWVzV5v4sSJvP7663Tv3p2//OUvfPvb3+Z3v/vdJ9qtWLGCJUuWcPDgQaZPn85nP/tZAJYvX87q1asZOnQos2bN4s033+Sss87itttu43vf+x4AV199NS+++CIXXXQRjzzyCABf+cpXjut933333SxYsIBzzz2Xe++9l549ex7X60U6vIO7Y3oaRbDtXThcFpxL7wf50+HMrwU9jfxTod/g5MbbjEQmkigwLOZ5AbCtcSMzOw+4G5jt7p+4vcDdt5nZauBTBPMoAAuBDe7+w2N9cXdfGLYjEok0mcDqtNRzSKQRI0Ywc+ZMXnzxRdasWcOsWbMAqKys5IwzzuCDDz5g9OjR9YvzrrzyShYuXHjM65WVlXHttdeyYcMGzIyqqqom211yySX07t2b3r17c8455/DOO++QnZ3NjBkzKCgIRgunTZvG5s2bOeuss1i0aBH3338/FRUV7Nmzh5NPPpmLLrrouBMIwA9+8AMGDx5MZWUl8+fP57777qtPUiKdUtVh2LGyIWlEl8K+cGNd6wYDT4aTP98wRJU7HrqlJTfm45DIRLIUGGdmo4Bi4ArgS7ENzGw68Cgw1913xRwvAErd/ZCZ9QdmAQ+F574PZBHc7dXh9e3bFwjmSM4//3x+85vfHHV++fLlx3W97373u5xzzjn84Q9/YPPmzcyZM6fJdo2Hpeqex/YM0tLSqK6u5vDhw9x6660UFRUxbNgw7rnnnlYtGBwyZEj917ruuuuanZgX6XDcYc+mo3sbO1ZCbfhLXb+hUHAqRK4PksaQadAzI7kxt1LCEom7V5vZbcCrBLfxPuHuq81sAVDk7i8ADwAZwLPhD7It7n4xcBLwH2bmBENkD7r7yjDB3A18ALwbvuY/3f2xRL2P9jJz5ky++tWvsnHjRsaOHUtFRQXRaJSJEyeyadMmNm/ezMiRI/ntb3/b7HXKysrIz88H4Be/+MUx2/3xj3/kX/7lXzh48CCLFy/m3nvvZf369U22rUsaubm5lJeX89xzz7Vqonz79u0MGTIEd+f5559n8uTJJ3wtkaSr2APFyxoSR/EyOLQ3ONejLwydDmfc2tDbyBza/PU6oISuI3H3l4GXGx37Xszj847xuv8BpjRxPErTcy8dXl5eHr/4xS+48sorOXIkGOH7/ve/z/jx4/npT3/K3Llzyc3NZcaMGc1e56677uLaa6/loYce4tOf/vQx282YMYPPfvazbNmyhe9+97sMHTr0mIkkOzubm266icLCQkaOHMlpp51Wfy52jmTHjh1EIhH2799Pt27d+OEPf8iaNWvIzMzkwgsv5LHHHmPo0KF8+ctfpqSkBHdn2rRp9dcQSXnVlUcPURUXBb0PAAwGngQTPxfOa0QgbyKkdf4tDc292emDTiESiXjjColr167lpJNOSlJEx6e8vJyMjAzcna9+9auMGzeOO+6444Svd88995CRkcE///M/t2GUbasjfX+kk3KHvR8Ft9zWD1GtgJpwdULG4IaJ8IJI0PPo2S+5MbcxM1vm7pGW2nX+VNkJ/PznP+epp56isrKS6dOnc/PNNyc7JJHO59DecIhqWcMQVUVpcK577yBRnH5zzBBVPmhfOEA9kiRF1HpPPvkkP/rR0UtoZs2axcMPP5ykiNpWR//+SIqrqYKdq8I7qMIhqtKN4UmDvAlhwjg1+HvgpC4xRNWYeiSd3HXXXcd1112X7DBEUp877NvSsDK8uChYLV4d3nnYNy9IFlOvgILTgp5Hr6zkxtzBKJGISOdyuAyK3z06cRwsCc517xWsCI/c0NDbyB6uIapWUiIRkY6rphp2rQ6Hp8JbcHevp34TjZxxMPa8hgnxQZMhTSWc25oSiYh0DO5QFo259XYZbHsPqg8F5/vkBD2MwkuDxJF/CvTun9yYuwglEhFJTUcOfHKIqnxncC4tPRiiOnVewy24/UdqiCpJlEi6iMWLF/Pggw8ec9fgeNs059e//jX33XcfABkZGfzsZz9j6tTE10KQTqC2BnatjdmLqghKPqB+iGrAGBg9p+FOqkGF0D09iQFLLCWSFNEZ6pGMGjWK1157jf79+/PKK68wf/583n777WSHJalo/7aja2xsWw5VB4NzvfsHPYxJlwR3UeWfAn0GJDdeaZYSCcAr3wq2PWhLgwvhgnubbdLZ6pGceeaZ9Y9nzpxJNBo9kX856WyOlMP2945OHAfCjcC79Qg+K9O/3LDQb8BoDVF1MEokSdZZ65E8/vjjXHCBqiB3ObU1ULLu6AnxXWvAa4Pz/UfCiDMb9qIaXAg9eiU1ZGk9JRJoseeQSJ2xHsmiRYt4/PHHeeONN1r7zyOp7sCORsWZ3oPKA8G5XlnBENWECxsmxPvmJjdeSQglkiTrbPVIVqxYwY033sgrr7xCTk7OccUuKa6y4pNDVPvD4ctu3YM1GlMvjxmiGpOU+uHS/pRIUkRnqEeyZcsWvvCFL/D0008zfvz4ON+5pKTaWijdcHRvY+dq8JrgfPZwGDYDCsI6G0OmQI/eyY1ZkkaJJEV0hnokCxYsoLS0lFtvvRWA7t2703izTElR5SVH19goXg5HwvrhPTOD/afOur2ht5ExMLnxSkpJ6O6/ZjYX+BFBhcTH3P3eRufvJCiZWw2UANe7+8dmNgL4ffi6HsBP3P2R8DWnAr8AehMUzfq6t/AmOvruv6pHIm2q6hBsX3F04ti3JThnaTBoUkPCyK+rH64hqq4o6bv/mlka8DBwPhAFlprZC+6+JqbZciDi7hVmdgtwP3A5sB04092PmFkGsCp87TbgZ8B8YAlBIpkLvJKo95EKVI9ETlhtLez5sNEQ1SqorQ7OZxYEC/xOu6mhfnh6n+TGLB1OIoe2ZgAb3X0TgJk9A1wC1CcSd18U034JcFV4vDLmeE+gW3iNIUCmu78VPv8l8I908kRyxx13fKIH0pp6JPfcc09bhiep5GBpoyGqZcFuuADpGcEQ1Zn/1NDj6Dc4ufFKp5DIRJIPbI15HgVOb6b9DcQkBDMbBrwEjAW+4e7bzCwSXif2mvlNXczM5hP0XBg+fHiTX9DdP3H3UkfRmeuRdIVia22i+sgnh6j2bg7OWbegGNOkf4ypHz4BuqUlNWTpnBKZSJr6Cd3kTwgzuwqIALPrG7pvBaaY2VDgeTN77niu6e4LgYUQzJE0Pt+rVy9KS0vJycnpsMmkM3J3SktL6dVLi9SO4g57NoVbpS8N64evhNpwnVC/ocEQ1anXNQxR9cxIbszSZSQykUSBYTHPC4BtjRuZ2XnA3cBsdz/S+HzYE1kNfAp4M7xOs9eMR0FBAdFolJKSkhN5uSRQr1696hdFdlkVe2J2vg2HqA7tCc716BMMUc28paG3kdVkx1ykXSQykSwFxpnZKKAYuAL4UmwDM5sOPArMdfddMccLgFJ3P2Rm/YFZwEPuvt3MDpjZTOBt4BrgJycSXI8ePepXi4skVXUl7FzZsFV6tCiYIAeC+uETYeKFDfMaeSd1yfrhkroS9r/R3avN7DbgVYLbeJ9w99VmtgAocvcXgAeADODZcHhpi7tfDJwE/IeZOcFw1oPuXrer4i003P77Cp18ol06GfdgHqOuml9xUTDPURN2xjMGBQmjbhPDodOhV2ZSQxZpSULXkaSKptaRiLSLQ/uCpFGfOJZBxe7gXPfeMHRaQxnY/AhkFWjnW0kZSV9HItLl1FQFazRi64eXbmg4nzsBxn+mIXEMnKT64dIpKJGInAj3YDV4bBnY7e9DdbiZZd+8oIdRt4lh/inBbrginZASiUg8Du+Hbe8e3ds4GN4fktYzqB8euSG4BTc/EmxqqCEq6SKUSEQaq6kOijHF9jZK1lG/ZClnLIz5dEONjUGTVT9cujQlEuna3GF/8dE1Nra/B1UVwfneA4KEcfIXwt7GqUFNcRGpp0QiXcuRA7Bt+dFDVOU7gnNp6TB4CpxyTbhm41ToP0pDVCItUCKRzqu2BnatPXp1eMkHDfXDB4yGUWfH1A+fDN17Nn9NEfkEJRLpPPZvO3qIattyqDoYnOuVHSSMky5umNvoMyC58Yp0Ekok0jFVHoRt7x3d29hfHJzr1gMGFzasDi+IBL0PDVGJJIQSiaS+ij3BQr8dq4K/t68I7qqqrx8+AobPbEgag6dAD+0eLNJelEgkddTWwt6Pgu3R6xLHjpWwP6YETd+BwVzGhDsbEkff3OTFLCJKJJIklRVBr2LHyobEsXM1VJYH5y0NcscFPY3BhUHyGFQI/QYlN24R+QQlEkksdziwI0wWYdLYsQpKN1K/wK9nZrCob9qXgqQxaDIMPAl69E5q6CISHyUSaTs1VbB7fUMvo66nUVHa0CZ7eDCHMfmLDT2N7BGaCBfpwJRI5MQc2tswh7Ez/LvkA6ipDM6n9Qx6FRMuDBNGIQw6WRsXinRCCU0kZjYX+BFBYavH3P3eRufvBG4EqoES4Hp3/9jMpgE/AzKBGuDf3f234WvOJSiI1Q0oB+a5+8ZEvo8urW4CPHbye+cqKNva0KZvXpAoxtwSzGMMngw541TFT6SLSNgn3czSgIeB8wnqty81sxfcfU1Ms+VAxN0rzOwW4H7gcqACuMbdN5jZUGCZmb3q7vsIEswl7r7WzG4FvgPMS9T76FJiJ8DrehlHTYB3g9zxMOx0OO1GTYCLCJDYHskMYKO7bwIws2eAS4D6ROLui2LaLwGuCo+vj2mzzcx2AXnAPoIZ2rrao1nAtgS+h86pbgJ85yrYsaKhp7Hnw4btQzQBLiJxSmQiyQdixj+IAqc30/4Gmqi/bmYzgHTgw/DQjcDLZnYI2A/MbJNoO6v6CfAwadT1NI45AT45SByaABeROCUykTT1U6jJAvFmdhUQAWY3Oj4EeBq41r3uV2XuAC5097fN7BvAQwTJpfE15wPzAYYPH36i76FjqZsAr0sWx5wAvyBIHIMmBxPgvbOTG7eIdGiJTCRRYFjM8wKaGIYys/OAu4HZ7n4k5ngm8BLwHXdfEh7LA6a6+9ths98Cf27qi7v7QmAhQCQSaTKBdVi1tbBvc8OajOYmwEd/JUgamgAXkQRJ5E+VpcA4MxsFFANXAF+KbWBm04FHgbnuvivmeDrwB+CX7v5szEv2AllmNj6cRzkfWJvA95B8lRXBVuj1w1JhjyN2AjxnXDgBfkM4n6EJcBFpPwlLJO5ebWa3Aa8S3P77hLuvNrMFQJG7v0BwG28G8KwF4/Fb3P1i4DLgbCDHzOaFl5zn7u+Z2U3A78ysliCxXJ+o99Cu3KF85ycX85VubDQBfnIwAT4onMvQBLiIJJm5d65Rn6ZEIhEvKipKdhgNaqpg94ZPbhtSsbuhTfbwcE1GoSbARSQpzGyZu0daaqcB80Q7tC9m8ntVkDh2rW1iAnyuJsBFpENSImkr9RPgsduGrIKyLQ1t6ibAT9cEuIh0HvoJdiLqJsB3xt41tRoqDwTn6yfAZ8Bp12sCXEQ6NSWS5tRPgDdazBc7AZ7eL+hZTLsynACfDAMnaQJcRLoMJZLm/NdlsOG/G57XTYCf/IVwn6lwC/Ru3ZIXo4hIkimRNGfypTDm3IYt0DUBLiLyCUokzZl6ebIjEBFJeRqTERGRVlEiERGRVlEiERGRVlEiERGRVlEiERGRVlEiERGRVlEiERGRVlEiERGRVlEiERGRVkloIjGzuWa2zsw2mtm3mjh/p5mtMbMVZvZXMxsRHp9mZm+Z2erw3OUxrzEz+3czW29ma83sa4l8DyIi0ryEbZFiZmnAwwR11aPAUjN7wd3XxDRbDkTcvcLMbgHuBy4HKoBr3H2DmQ0FlpnZq+6+D5gHDAMmunutmQ1M1HsQEZGWJbJHMgPY6O6b3L0SeAa4JLaBuy9y94rw6RKgIDy+3t03hI+3AbuAvLDdLcAC92Afd3fflcD3ICIiLTiuRGJm3cwsM87m+cDWmOfR8Nix3AC80sTXnAGkAx+Gh8YAl5tZkZm9YmbjjhHr/LBNUUlJSZwhi4jI8WoxkZjZf5lZppn1BdYA68zsG3Fc25o45sf4GlcBEeCBRseHAE8D19X1QICewOGwIP3PgSeauqa7L3T3iLtH8vLymmoiIiJtIJ4eySR33w/8I/AyMBy4Oo7XRQnmMuoUANsaNzKz84C7gYvd/UjM8UzgJeA77r6k0XV/Fz7+AzAljlhERCRB4kkkPcysB0Ei+aO7V3GMnkUjS4FxZjbKzNKBK4AXYhuY2XTgUYIksivmeDpBkviluz/b6LrPA58OH88G1scRi4iIJEg8ieRRYDPQF3g9vEV3f0svcvdq4DbgVWAt8P+5+2ozW2BmF4fNHgAygGfN7D0zq0s0lwFnA/PC4++Z2bTw3L3AF81sJfAD4MZ43qiIiCSGucfTuWj0IrPuYaLoECKRiBcVFSU7DBGRDsXMloXz0c2KZ7L96+Fku5nZ42b2Lg1DSyIi0sXFM7R1fTjZ/g8EazmuIxheEhERiSuR1N3GeyHwpLu/T9O39oqISBcUTyJZZmb/TZBIXjWzfkBtC68REZEuIp69tm4ApgGbwj2xcgiGt0RERFpOJOHGiAXAl8wM4DV3/1PCIxMRkQ4hnru27gW+TrA9yhrga2b2g0QHJiIiHUM8Q1sXAtPq9roys6cItn//l0QGJiIiHUO8u/9mxzzOSkQgIiLSMcXTI/kBsNzMFhHc9ns26o2IiEgonsn235jZYuA0gkTyTVTrXUREQnGV2nX37cTs3GtmWwi2kxcRkS7uRHsWWtkuIiLAiSeS498yWEREOqVjDm2Z2U9oOmEYR9/FJSIiXVhzPZIiYFkTf4qAf4rn4mY218zWmdlGM/tWE+fvNLM1ZrbCzP4aFs3CzKaZ2Vtmtjo8d3kTr/2JmZXHE4eIiCTOMXsk7v5Uay5sZmnAw8D5BHXWl5rZC+6+JqbZciAS7uF1C3A/cDlQAVzj7hvMbCjBxpGvuvu+8NoR1CsSEUkJibyNdwaw0d03uXsl8AxwSWwDd1/k7hXh0yVAQXh8vbtvCB9vA3YR1EKpS1APAHclMHYREYlTIhNJPrA15nk0PHYsNwCvND5oZjOAdODD8NBtwAvhLckiIpJkca0jOUFN3SLc5N1eZnYVEAFmNzo+BHgauDbchXgo8L+AOS1+cbP5wHyA4cO15EVEJFFaTCRm9uMmDpcBRe7+x2ZeGgWGxTwvALY1cf3zgLuB2e5+JOZ4JvAS8B13XxIeng6MBTaGW9r3MbON7j628XXdfSGwECASieh2ZRGRBIlnaKsXQWGrDeGfKcAA4AYz+2Ezr1sKjDOzUWaWDlxBzOp4ADObDjwKXOzuu2KOpwN/AH7p7s/WHXf3l9x9sLuPdPeRQEVTSURERNpPPENbY4FPu3s1gJn9DPhvgruxVh7rRe5ebWa3Aa8CacAT7r7azBYQ9GZeIJg0zwCeDXsYW9z9YuAygs0hc8xsXnjJee7+3gm8RxERSaB4Ekk+0JdgOIvw8VB3rzGzI8d+Gbj7y8DLjY59L+bxecd43a+AX7UUmLtntNRGREQSK55Ecj/wXrgDcN028v/bzPoCf0lgbCIi0gHEs43842b2MsG6EAO+Ha7tAPhGIoMTEZHUF+/tv6cBnwof19DE3VciItI1tXjXlpndC3wdWBP++ZqZ/SDRgYmISMcQT4/kQmCau9cCmNlTBHtkqdyuiIjEvUVK7AaJWYkIREREOqZ4eiQ/AJab2SIa7tpSb0RERID47tr6TXjr72kEieSbJHazRxER6UDiumsr3Gm3fnsTM9sCaCdEERE54Z5FUzv7iohIF3SiiUS76YqICNDM0JaZ/YSmE4ahMrciIhJqbo6k6ATPiYhIF3LMROLuTzU+ZmaD3X1HYkMSEZGO5HjnSF5uuYmIiHQlx5tIdLeWiIgc5XgTyc+Pp7GZzTWzdWa20cy+1cT5O81sjZmtMLO/mtmI8Pg0M3vLzFaH5y6Pec2vw2uuMrMnzKzHcb4HERFpQ8eVSNz9p/G2NbM04GHgAmAScKWZTWrUbDkQcfcpwHMERbQAKoBr3P1kYC7wQzOru1Ps18BEoBDoDdx4PO9BRETaViK3OpkBbHT3Te5eCTwDXBLbwN0XuXtF+HQJUBAeX+/uG8LH24BdQF74/GUPAe/UvUZERJIjkYkkH9ga8zwaHjuWG4BXGh80sxlAOvBho+M9gKuBPzd1MTObb2ZFZlZUUlJynKGLiHR81TW1BL9zJ1a8FRJPRFMT802+IzO7CogAsxsdHwI8DVxbVw8lxk+B1939b01d090XAgsBIpGIVuKLSKdWU+t8tLuc97eWsbK4jPej+1izbT+LvzGHIVm9E/q1E5lIosCwmOcFNFGi18zOA+4GZrv7kZjjmcBLwHfcfUmj1/wrwVDXzQmIW0Qkpbk7W/ZU8H60jJXRfayIlrGquIyDlTUA9O6RxuT8TK6aOaJd4klkIlkKjDOzUUAxcAXwpdgGZjYdeBSY6+67Yo6nA38AfunuzzZ6zY3AZ4Bzm+iliIh0Ku7O9rLDrAgTxsriMlZEyyg7VAVAelo3ThqayRdPLaAwP4upw7IZk5dBWrf2W62RsETi7tVmdhvwKpAGPOHuq81sAVDk7i8ADwAZwLNmBrDF3S8GLiMooJVjZvPCS85z9/eAR4CPgbfC1/ze3Rck6n2IiLSnkgNHWFm8r36IakV0H7vLKwFI62ZMGNSPCwsHU5ifzZSCLMYP6kd69+SWiLL2mIhJtkgk4kVF2h5MRFLLvorK+h5GXY9je9lhAMxgbF4GUwqChFFYkMWkIZn06pHWbvGZ2TJ3j7TULpFDWyIiEio/Us2q4oaEsSJaxpY9FfXnR+b04bSRA4KkkZ/F5Pws+vbsGD+iO0aUIiIdyOGqGlZv28+K6D5WRoM7qDbtPkjdAFB+dm+mFGRxxYxhTMnPpjA/i6w+HXeTDiUSEZFWqKyuZd2OA7wfJo0VxWWs33mAmtoga+Rm9GRqQRYXT82vH6LKzeiZ5KjblhKJiEicqmtq2VhSXj+nsTJaxtrtB6isCW4gze7Tg8L8LM6dOIbCgiymFmQzKLMn4Y1BnZYSiYhIE2prnY9KD9YPTa2MlrF6234OVQVrNTJ6dmdyfibXzRpZnzQK+vfu9EmjKUokItLluTvRvYeOuntqVXEZB45UA9CrRzdOHhrOaRRkMaUgm1E5fenWjms1UpkSiYh0OTtiFvitKA5Wh++tCBb49UgzThqSySXThwYT4QVZjBuYQfe05K7VSGVKJCLSqZWWH2FFcRkrtpaxsjhIHrsOBLsxpXUzxg3M4B8mDaawIIspBVlMGNyPnt3bb61GZ6BEIiKdRtmhqnCtRsMQVfG+Q0CwwG90bl9mjc0Nh6eymDQki97pShqtpUQiIh3SwSPV9Ws16vag+mj3wfrzwwf0YdrwbK49cwSF+dlMzs+kX6+Ou1YjlSmRiEjKO1xVw9rt+4Pt0cMhqo27ygmXajAkqxf3RKKrAAAVJ0lEQVSF+VlcGm5cOKUgi+w+6ckNugtRIhGRlFJVEyzwWxmznci6HQeoDrNGTt90phRkccHkIfUL/Ab265XkqLs2JRIRSZqaWufDmAV+K6JlrNm+n8rqYIFfZq/uTCnIZv7Zo+tvux2S1atLrtVIZUokItIu3J3NpRX1K8JXRMtYta2MirAYU5/0NCbnZ3HtGSMoLMhmSn4WI3L6KGl0AEokItLm3J1tZYdZsXVfcOttmDz2Hw4W+PXs3o1JQzO5LDKsfk5jdDsXY5K2k9BEYmZzgR8RFLZ6zN3vbXT+TuBGoBooAa5394/NbBrwMyATqAH+3d1/G75mFPAMMAB4F7ja3SsT+T5EpHm7Dhxmxdayo5JG6cHgY9m9mzFxSD8+N3UoU/KDOY3xg/rRQwv8Oo2EJRIzSwMeBs4nqN++1MxecPc1Mc2WAxF3rzCzW4D7gcuBCuAad99gZkOBZWb2qrvvA+4D/o+7P2NmjwA3ECQdEWkHew9W1q8GD2qGl7Fjf1CMqZvBuIH9+PTEgeFEeDYTB/dr12JM0v4S2SOZAWx0900AZvYMcAlQn0jcfVFM+yXAVeHx9TFttpnZLiDPzMqAT9NQ+/0p4B6USEQS4sDhKlYWl9Vvj74iuo+tew7Vnx+d25fTRw+or+J38tBM+qRrxLyrSeR3PB/YGvM8CpzeTPsbgFcaHzSzGUA68CGQA+xz9+qYa+a3SbQiXdyhyhpWbyurX9z3fnQfm0oaFvgV9A+KMX359BFMyc9ickEWmVrgJyQ2kTQ1a9ZkgXgzuwqIALMbHR8CPA1c6+611vTtG8e65nxgPsDw4cOPI2yRzu9IdQ0fbD9QP0S1IhoUY6pb4DcosyeF+dl8flo+hWHp15xOVoxJ2k4iE0kUGBbzvADY1riRmZ0H3A3MdvcjMcczgZeA77j7kvDwbiDbzLqHvZImrwng7guBhQCRSKTJZCPSFVTX1LJhV/lRtcI/2LGfqprgY9G/Tw+mFGTzD5MGBbfdFmQxKFML/CR+iUwkS4Fx4V1WxcAVNMxtAGBm04FHgbnuvivmeDrwB+CX7v5s3XF3dzNbBFxKcOfWtcAfE/geRDqU2lpn0+6DMUljH2u27+dwVbDAr1/P7hQWZHHDWcECv8L8rC5bjEnaTsISibtXm9ltwKsEt/8+4e6rzWwBUOTuLwAPABnAs+F/5C3ufjFwGXA2kGNm88JLznP394BvAs+Y2fcJ7vp6PFHvQSSVuTtb9xwKqveFE+GrivdTHhZj6t0jjcn5mcGcRpg0RqoYkySAuXf+UZ9IJOJFRUXJDkPkhLk7O/Yfrt+wsG5CfF9YjCk9rRsnDc1kSri4b0pBNmMHaoGftI6ZLXP3SEvtdJ+eSAraXX7kqFrh70fL2F3eUIxpwqB+zD15cP1tt+MH9SO9uxb4SXIokYgkWVlFVf3ttivDeY1tZcECPzMYm5fB2eNzmVoQlH2dNCRTC/wkpSiRiLSj8iPVrK6r4Bfeeru5tKL+/MicPpw6cgDXh3MaJ+dnkdFTH1NJbfofKpIgh6tqWLN9f8zGhWV8WFJO3bRkfnZvCvOzuOy0YUzJz6YwP4usPlrgJx2PEolIG6isrmX9zgNH1dVYv7OhGFNuRk+mFmRx0ZShTCnIYnJ+Fnn9tMBPOgclEpHjVFPrbIxd4FdcxtqYYkzZfXpQmJ/FzRNH10+GD85UMSbpvJRIRJpRW+tsLj14VK3wVcX7OVQVFGPK6NmdyfmZzDtzZHDbbX42wwZogZ90LUokIiF3J7r30FF3UK0sLuNAWIypV49unDw0i8tPG8bUYVkU5mczOlcL/ESUSKTL2rn/8FFzGiuLy9gTFmPqkWacNCSTi6cOrb/tdtzADLqrGJPIJyiRSJew52BlfeW+96PBENXO/Q0L/MYNzOC8kwbWz2lMGNyPnt21VkMkHkok0unsP1zFqphCTCuiZUT3BsWYzIJiTGeOyQ23Esli0pAseqcraYicKCUS6dAqKqtZvW1//RDVymgZm3Y3FGMaPqAPU4dlc80ZIyjMz2Zyfib9VIxJpE0pkUiH4e5sLq3gjY27eX9rkDQ27GooxjQkqxeF+Vl88dQCCvODleH9+6YnN2iRLkCJRFLawSPVvPVhKa+tL+G19SVs2RNsJ5LTN50pBVl8ZvJgphZkUViQxcB+KsYkkgxKJJJS3J0Nu8pZvG4Xr60vYelHe6msqaVPehpnjsnhpk+N4uzxeQwf0EdrNURShBKJJN3+w1X8feNuFq8Leh3bw51vJwzqx7xZI5k9Po/IyP66i0okRSU0kZjZXOBHBBUSH3P3exudvxO4EagGSoDr3f3j8NyfgZnAG+7+uZjXnEtQWbEbUE5QOXFjIt+HtK3aWmfN9v3BcNW6EpZt2UtNrdOvZ3fOGpfL18/NY/aEPIZk9U52qCISh4QlEjNLAx4GzgeiwFIze8Hd18Q0Ww5E3L3CzG4B7gcuD889APQBbm506Z8Bl7j7WjO7FfgOMC9R70Paxt6Dlfxt424Wr9vF6+t31xdpmpyfyVdmj2b2+IFMH55NDy34E+lwEtkjmQFsdPdNAGb2DHAJUJ9I3H1RTPslwFUx5/5qZnOauK4DmeHjLGBb24YtbaGm1lkR3cdr60tYvK6E96P7cA82NDx7XB6zx+fxqfG5miAX6QQSmUjyga0xz6PA6c20vwF4JY7r3gi8bGaHgP0Ew1+fYGbzgfkAw4cPjydeaaWSA0d4fX0Ji9eX8LcNJeyrqMIMpg3L5uvnjmP2+DymFGSrjrhIJ5PIRNLUTwtvsqHZVUAEmB3Hde8ALnT3t83sG8BDBMnl6C/kvhBYCBCJRJr8utI6VTW1LN+yj9fW72LxuhJWb9sPBLU3zp04iNkT8vjU2Fyt5RDp5BKZSKLAsJjnBTQxDGVm5wF3A7Pd/UhzFzSzPGCqu78dHvot8Oe2CVfisb3sEK+Fd1e9sXE3Bw5Xk9bNOHV4f77xmQnMHp/HpCGZ2hFXpAtJZCJZCowzs1FAMXAF8KXYBmY2HXgUmOvuu+K45l4gy8zGu/t6gon8tW0btsQ6Ul1D0ea99XdYrdt5AAhWkX+2cAhzJuRx5thcMrXtiEiXlbBE4u7VZnYb8CrB7b9PuPtqM1sAFLn7CwR3ZmUAz4aLy7a4+8UAZvY3YCKQYWZR4AZ3f9XMbgJ+Z2a1BInl+kS9h65q656K+gWBf/+wlIrKGtLTunHaqP588dSJzJkwkHEDM7QgUEQAMPfOP30QiUS8qKgo2WGkrMNVNby1qZTX1pXw+vqS+k0Phw3ozZzxA5k9Po8zxuTQt6fWr4p0JWa2zN0jLbXTT4YuyN3ZtPtg/UrytzeVcqS6lp7du3HGmByuPmMEs8fnMSq3r3odItIiJZIuojzc/LBuyKquPseYvL58+fQRzJ6Qx+mjBtCrh7YhEZHjo0TSSbk763Ye4LV1wYLAoo/3UFXj9E1P48yxuXxl9hhmj89j2IA+yQ5VRDo4JZJOpOxQFW+G25C8tr6kvpTsxMH9uP6sUcHmhyMGkN5d25CISNtRIunAamud1dv21y8IXL51X7D5Ya/u9duQnD0+j8FZ2oZERBJHiaSD2XOwkr9tCNZ0vL6hhN3llQAU5mdx65xguGrasGy6a/NDEWknSiQprqbWeW/rvnBB4C5WFJfhDv379ODs8XnMmZDHp8blkZvRM9mhikgXpUSSgnbtP1xfWvZvG3ZTdqiKbuHmh7efO545E/KYnJ+lzQ9FJCUokaSAqppaln28t37L9bXbg80P8/r15PxJg4It18flkt1Hmx+KSOpRIkmS4n11mx/u4s2NpZQfqaZ7N+PUEf25a+4E5owfyElD+mlBoIikPCWSdnK4qoalm/fU75y7YVc5AEOzenHR1KHMHp/HrLE59NPmhyLSwSiRJNDHpQ3bkLz1YSmHqoLND08fPYDLTxvG7PF5jNXmhyLSwSmRtKFDlTUs2dSwDcnm0goARuT04bJIAbMn5DFzdA590vXPLiKdh36itYK782FJecPmhx/tobK6ll49unHG6ByumxWsJh+Z2zfZoYqIJIwSyXE6cLiKv39YyuJwy/XifcHmh2MHZnD1zBHMmZDHaSO1+aGIdB0JTSRmNhf4EUFhq8fc/d5G5+8kqLdeDZQA17v7x+G5PwMzgTfc/XMxrzHg+8D/AmqAn7n7jxP1HtydtdsPhLfm7mLZx3uprg02P5w1NpevnjOWs8fnUtBfmx+KSNeUsERiZmnAwwTlcKPAUjN7wd3XxDRbDkTcvcLMbgHuBy4Pzz0A9AFubnTpeQS14Ce6e62ZDUzUe7j/zx/w3LIouw4Emx+eNCSTGz81mjkT8jhleH9tfigiQmJ7JDOAje6+CcDMngEuAeoTibsvimm/BLgq5txfzWxOE9e9BfiSu9eG7eKp9X5Camqd00YNYPb4YAPEQZna/FBEpLFEJpJ8YGvM8yhwejPtbwBeieO6Y4DLzezzBMNhX3P3DSccZTP+5cKTEnFZEZFOJZFjM00tjmiyQLyZXQVECIazWtITOBzWEf458MQxrjnfzIrMrKikpCTOkEVE5HglMpFECeYy6hQA2xo3MrPzgLuBi939SJzX/V34+A/AlKYauftCd4+4eyQvL++4AhcRkfglMpEsBcaZ2SgzSweuAF6IbWBm04FHCZJIvHMdzwOfDh/PBta3UbwiInICEjZH4u7VZnYb8CrB7b9PuPtqM1sAFLn7CwRDWRnAs+E2IVvc/WIAM/sbMBHIMLMocIO7vwrcC/zazO4AygluHxYRkSQx9yanLTqVSCTiRUVFyQ5DRKRDMbNl4Xx0s7QQQkREWkWJREREWkWJREREWqVLzJGYWQnwcbLjCOUCu5MdRAtSPcZUjw8UY1tI9fgg9WNsbXwj3L3F9RNdIpGkEjMrimfyKplSPcZUjw8UY1tI9fgg9WNsr/g0tCUiIq2iRCIiIq2iRNL+FiY7gDikeoypHh8oxraQ6vFB6sfYLvFpjkRERFpFPRIREWkVJZIEMbO5ZrbOzDaa2beO0eYyM1tjZqvN7L9SLUYzG25mi8xsuZmtMLML2zm+J8xsl5mtOsZ5M7Mfh/GvMLNTUiy+L4dxrTCzv5vZ1PaML54YY9qdZmY1ZnZpe8UWft0W4zOzOWb2Xvg5ea094wu/fkvf5ywz+5OZvR/GeF07xzcs/JyuDb/+15tok9jPirvrTxv/Idik8kNgNJAOvA9MatRmHEGp4f7h84EpGONC4Jbw8SRgczvHeDZwCrDqGOcvJCiGZsBM4O0Ui+/MmO/vBe0dXzwxxvxf+L/Ay8ClqRQfkE1QVXV4+LxdPydxxvht4L7wcR6wB0hvx/iGAKeEj/sR7Ije+LOc0M+KeiSJUV9m2N0rgboyw7FuAh52972Q2JLBrYjRgczwcRZN1JNJJHd/neBDeSyXAL/0wBIg28yGtE90Lcfn7n+v+/4SlJIuaJfAjo6hpX9DgH8iqPHT3v8H44nvS8Dv3X1L2D4VY3SgnwVbmGeEbavbIzYAd9/u7u+Gjw8Aawkq1MZK6GdFiSQxmioz3PgbOx4Yb2ZvmtkSM5vbbtEF4onxHuCqcBv/lwl+4KSSeN5Dqoi3lHS7MrN84PPAI8mO5RjGA/3NbLGZLTOza5IdUBP+EziJ4BetlcDX3b02GYGY2UhgOvB2o1MJ/awksmZ7VxZPmeHuBMNbcwh+U/2bmU12930Jjq1OPDFeCfzC3f/DzM4Ang5jTMqHpAlxl3NOJjM7hyCRnJXsWJrwQ+Cb7l4T1gRKNd2BU4Fzgd7AW2a2xN1TqaDdZ4D3CArujQH+x8z+5u772zMIM8sg6Fne3sTXTuhnRYkkMeIpMxwFlrh7FfCRma0jSCxL2yfEuGK8AZgL4O5vmVkvgr172n144RjiKuecTGY2BXgMuMDdS5MdTxMiwDNhEskFLjSzand/Prlh1YsCu939IHDQzF4HppJalVGvA+71YDJio5l9RFCU7532CsDMehAkkV+7+++baJLQz4qGthKjxTLDBCWDzwEws1yCLvymFItxC8FvgpjZSUAvoKQdY2zJC8A14R0pM4Eyd9+e7KDqmNlw4PfA1Sn2G3Q9dx/l7iPdfSTwHHBrCiURgD8CnzKz7mbWBzidYA4glcR+TgYBE2jHz3I4N/M4sNbdHzpGs4R+VtQjSQCPr8zwq8A/mNkaoAb4Rnv+xhpnjP8P8HMLyho7MC/8ratdmNlvCIb+csN5mn8FeoTxP0Iwb3MhsBGoIPjNsN3EEd/3gBzgp+Fv/NXezhv8xRFjUrUUn7uvNbM/AyuAWuAxd2/2Vub2jhH4N+AXZraSYAjpm+7enjsCzwKuBlaa2XvhsW8Dw2NiTOhnRSvbRUSkVTS0JSIiraJEIiIiraJEIiIiraJEIiIiraJEIiIiraJEIiIiraJEIpIiwu3SX2xtG5H2pkQiEqdwVbA+MyKN6EMh0gwzGxkWDPop8C5wtZm9ZWbvmtmz4UZ5mNmFZvaBmb0RFhA6Zq/BzGZYUOhqefj3hCba3GNmT5vZ/zWzDWZ2U8zpDDN7Lvx6vw63yMDMvmdmS81slZktrDsukmhKJCItmwD8EjifYCPL89z9FKAIuDPczPJRgo0ZzyIobtScD4Cz3X06wTYq//sY7aYAnwXOAL5nZkPD49OB2wmKjY0m2CID4D/d/TR3n0ywU+7njvudipwA7bUl0rKP3X2JmX2O4If3m+Ev++nAWwQ7vW5y94/C9r8B5jdzvSzgKTMbR7CHWY9jtPujux8CDpnZIoJiZPuAd9w9ChDurTQSeAM4x8zuAvoAA4DVwJ9O7C2LxE+JRKRlB8O/Dfgfd78y9qSZTT/O6/0bsMjdPx8WIlp8jHaNN8Kre34k5lgN0D3sFf0UiLj7VjO7h2C3ZpGE09CWSPyWALPMbCyAmfUxs/EEQ1Wjw6QAcHkL18kCisPH85ppd4mZ9TKzHILdZ5urVVOXNHaH8zaXthCDSJtRIhGJk7uXEPzg/42ZrSBILBPD4adbgT+b2RvATqCsmUvdD/zAzN4k2ML/WN4BXgq/zr+5+zELEYWVNX9OUOr1edqvQJqItpEXaQtmluHu5eGdUg8DG9z9/7TievcA5e7+YFvFKJIo6pGItI2bwonv1QRDV48mOR6RdqMeiUiCmNl1wNcbHX7T3b+ajHhEEkWJREREWkVDWyIi0ipKJCIi0ipKJCIi0ipKJCIi0ipKJCIi0ir/P4imEFSjBiZEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x241994c5940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch5_1.best_score_, gsearch5_1.best_params_))\n",
    "test_means = gsearch5_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch5_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch5_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch5_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch5_1.cv_results_).to_csv('my_preds_reg_alpha_reg_lambda_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(reg_alpha), len(reg_lambda))\n",
    "train_scores = np.array(train_means).reshape(len(reg_alpha), len(reg_lambda))\n",
    "\n",
    "#log_reg_alpha = [0,0,0,0]\n",
    "#for index in range(len(reg_alpha)):\n",
    "#   log_reg_alpha[index] = math.log10(reg_alpha[index])\n",
    "    \n",
    "for i, value in enumerate(reg_alpha):\n",
    "    pyplot.plot(reg_lambda, -test_scores[i], label= 'reg_alpha:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'reg_alpha' )                                                                                                      \n",
    "pyplot.ylabel( '-Log Loss' )\n",
    "pyplot.savefig( 'reg_alpha_vs_reg_lambda1.png' )"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处使用GridSearchCV实现了对reg_alpha和reg_lambda参数的调整，并且得到了最佳参数为1.5和0.5，最佳结果为-0.216146。\n",
    "因为计算资源有限，所以暂时不进行对上述两个参数的进一步调整。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作业总结"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "内容小结：\n",
    "1）本次作业只是对老师提供的代码在不同数据集上进行了复现，并未尝试其他函数；\n",
    "2）因为计算机性能的原因，并没有对各个参数进行更为细致的调整；\n",
    "3）熟悉了xgboost的使用方法，对数据集的处理和训练更加巩固了这一知识；\n",
    "4）在使用和学习过程中，发现xgboost相较于GDBT学习得到的模型更加简单，可以有效防止过拟合；\n",
    "5）本次作业因为每部分之间的运行过程较长，因而并未按照作业说明的顺序来做，但是已经熟悉了模型的使用和相应参数的调整方法。\n",
    "\n",
    "问题小结：\n",
    "1）从每次运行的结果来看，所得到的结果似乎都不太理想，我暂时还想不到具体问题出在哪里；\n",
    "2）关于运行时间问题，我每段代码的运行时间过长，部分代码甚至运行时间超过一天，这个有没有可能是代码的问题；\n",
    "3）虽然对xgboost模型已经理解，但是对其适用的数据集类型还不太了解。\n",
    "\n",
    "最后，因为个人原因和设备问题，没有按照制定时间提交作业，向各位助教致以最诚挚的道歉。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
